Data Model Logger - Data Discovery for Extract-Transform-Load

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

Researchers

Research units

Abstract

Information Systems (ISs) are fundamental to streamline operations and support processes of any modern enterprise. Being able to perform analytics over the data managed in various enterprise ISs is becoming increasingly important for
organisational growth. Extract, Transform, and Load (ETL) are the necessary pre-processing steps of any data mining activity. Due to the complexity of modern IS, extracting data is becoming increasingly complicated and time-consuming. In order to ease the process, this paper proposes a methodology and a pilot implementation, that aims to simplify data extraction process by leveraging the end-users’ knowledge and understanding of the specific IS. This paper first provides a brief introduction and the current state of the art regarding existing ETL process and techniques. Then, it explains in details the proposed methodology. Finally, test results of typical data-extraction tasks from 4 commercial ISs are reported.

Details

Original languageEnglish
Title of host publication2017 IEEE 19th International Conference on High Performance Computing and Communications; IEEE 15th International Conference on Smart City; IEEE 3rd International Conference on Data Science and Systems (HPCC/SmartCity/DSS)
Publication statusPublished - 2018
MoE publication typeA4 Article in a conference publication
EventIEEE International Conference on Data Science and Systems - Bangkok, Thailand
Duration: 18 Dec 201720 Dec 2017
Conference number: 3

Conference

ConferenceIEEE International Conference on Data Science and Systems
Abbreviated titleDSS
CountryThailand
CityBangkok
Period18/12/201720/12/2017

    Research areas

  • ETL, Database, Trigger, Reverse Engineering, Data Warehouse, Information System, Information Retrieval, Process Mapping, Data Discovery

ID: 16075728