Data discovery method for Extract- Transform-Load

Manik Madhikerrni; Kary Främling

doi:10.1109/ICMIMT.2019.8712027

Data discovery method for Extract- Transform-Load

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

3 Citations (Scopus)

243 Downloads (Pure)

Abstract

Information Systems (ISs) are fundamental to streamline operations and support processes of any modern enterprise. Being able to perform analytics over the data managed in various enterprise ISs is becoming increasingly important for organisational growth. Extract, Transform, and Load (ETL) are the necessary pre-processing steps of any data mining activity. Due to the complexity of modern IS, extracting data is becoming increasingly complicated and time-consuming. In order to ease the process, this paper proposes a methodology and a pilot implementation, that aims to simplify data extraction process by leveraging the end-users' knowledge and understanding of the specific IS. This paper first provides a brief introduction and the current state of the art regarding existing ETL process and techniques. Then, it explains in details the proposed methodology. Finally, test results of typical data-extraction tasks from four commercial ISs are reported.

Original language	English
Title of host publication	2019 IEEE 10th International Conference on Mechanical and Intelligent Manufacturing Technologies, ICMIMT 2019
Publisher	IEEE
Pages	174-181
Number of pages	8
ISBN (Electronic)	9781538679722
DOIs	https://doi.org/10.1109/ICMIMT.2019.8712027
Publication status	Published - 9 May 2019
MoE publication type	A4 Conference publication
Event	IEEE International Conference on Mechanical and Intelligent Manufacturing Technologies - Cape Town, South Africa Duration: 15 Feb 2019 → 17 Feb 2019 Conference number: 10

Conference

Conference	IEEE International Conference on Mechanical and Intelligent Manufacturing Technologies
Abbreviated title	ICMIMT
Country/Territory	South Africa
City	Cape Town
Period	15/02/2019 → 17/02/2019

Keywords

Data Discovery
Data Warehouse
Database
ETL
Information Retrieval
Information System
Process Mapping
Reverse Engineering
Trigger

Access to Document

10.1109/ICMIMT.2019.8712027

SCI_Främling_Data_Discovery.MT0003Accepted author manuscript, 308 KB

Cite this

@inproceedings{f708f4ddadc04e208868e9a755fda5e7,

title = "Data discovery method for Extract- Transform-Load",

abstract = "Information Systems (ISs) are fundamental to streamline operations and support processes of any modern enterprise. Being able to perform analytics over the data managed in various enterprise ISs is becoming increasingly important for organisational growth. Extract, Transform, and Load (ETL) are the necessary pre-processing steps of any data mining activity. Due to the complexity of modern IS, extracting data is becoming increasingly complicated and time-consuming. In order to ease the process, this paper proposes a methodology and a pilot implementation, that aims to simplify data extraction process by leveraging the end-users' knowledge and understanding of the specific IS. This paper first provides a brief introduction and the current state of the art regarding existing ETL process and techniques. Then, it explains in details the proposed methodology. Finally, test results of typical data-extraction tasks from four commercial ISs are reported.",

keywords = "Data Discovery, Data Warehouse, Database, ETL, Information Retrieval, Information System, Process Mapping, Reverse Engineering, Trigger",

author = "Manik Madhikerrni and Kary Fr{\"a}mling",

note = "| openaire: EC/H2020/688203/EU//BIoTope ; IEEE International Conference on Mechanical and Intelligent Manufacturing Technologies, ICMIMT ; Conference date: 15-02-2019 Through 17-02-2019",

year = "2019",

month = may,

day = "9",

doi = "10.1109/ICMIMT.2019.8712027",

language = "English",

pages = "174--181",

booktitle = "2019 IEEE 10th International Conference on Mechanical and Intelligent Manufacturing Technologies, ICMIMT 2019",

publisher = "IEEE",

address = "United States",

}

Madhikerrni, M & Främling, K 2019, Data discovery method for Extract- Transform-Load. in 2019 IEEE 10th International Conference on Mechanical and Intelligent Manufacturing Technologies, ICMIMT 2019., 8712027, IEEE, pp. 174-181, IEEE International Conference on Mechanical and Intelligent Manufacturing Technologies, Cape Town, South Africa, 15/02/2019. https://doi.org/10.1109/ICMIMT.2019.8712027

TY - GEN

T1 - Data discovery method for Extract- Transform-Load

AU - Madhikerrni, Manik

AU - Främling, Kary

N1 - Conference code: 10

PY - 2019/5/9

Y1 - 2019/5/9

N2 - Information Systems (ISs) are fundamental to streamline operations and support processes of any modern enterprise. Being able to perform analytics over the data managed in various enterprise ISs is becoming increasingly important for organisational growth. Extract, Transform, and Load (ETL) are the necessary pre-processing steps of any data mining activity. Due to the complexity of modern IS, extracting data is becoming increasingly complicated and time-consuming. In order to ease the process, this paper proposes a methodology and a pilot implementation, that aims to simplify data extraction process by leveraging the end-users' knowledge and understanding of the specific IS. This paper first provides a brief introduction and the current state of the art regarding existing ETL process and techniques. Then, it explains in details the proposed methodology. Finally, test results of typical data-extraction tasks from four commercial ISs are reported.

AB - Information Systems (ISs) are fundamental to streamline operations and support processes of any modern enterprise. Being able to perform analytics over the data managed in various enterprise ISs is becoming increasingly important for organisational growth. Extract, Transform, and Load (ETL) are the necessary pre-processing steps of any data mining activity. Due to the complexity of modern IS, extracting data is becoming increasingly complicated and time-consuming. In order to ease the process, this paper proposes a methodology and a pilot implementation, that aims to simplify data extraction process by leveraging the end-users' knowledge and understanding of the specific IS. This paper first provides a brief introduction and the current state of the art regarding existing ETL process and techniques. Then, it explains in details the proposed methodology. Finally, test results of typical data-extraction tasks from four commercial ISs are reported.

KW - Data Discovery

KW - Data Warehouse

KW - Database

KW - ETL

KW - Information Retrieval

KW - Information System

KW - Process Mapping

KW - Reverse Engineering

KW - Trigger

UR - http://www.scopus.com/inward/record.url?scp=85066473340&partnerID=8YFLogxK

U2 - 10.1109/ICMIMT.2019.8712027

DO - 10.1109/ICMIMT.2019.8712027

M3 - Conference article in proceedings

AN - SCOPUS:85066473340

SP - 174

EP - 181

BT - 2019 IEEE 10th International Conference on Mechanical and Intelligent Manufacturing Technologies, ICMIMT 2019

PB - IEEE

T2 - IEEE International Conference on Mechanical and Intelligent Manufacturing Technologies

Y2 - 15 February 2019 through 17 February 2019

ER -

Data discovery method for Extract- Transform-Load

Abstract

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this