Exploiting data reduction principles in cloud-based data management for cryo-image data

Kashish Ara Shakil; Mansaf Alam; Shabih Shakeel; Ari Ora; Samiya Khan

doi:10.1145/3232174.3232177

Exploiting data reduction principles in cloud-based data management for cryo-image data

Kashish Ara Shakil, Mansaf Alam, Shabih Shakeel, Ari Ora, Samiya Khan

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Scientific › peer-review

2 Citations (Scopus)

Abstract

Cloud computing is a cost-effective way for start-up life sciences laboratories to store and manage their data. However, in many instances the data stored over the cloud could be redundant which makes cloud-based data management inefficient and costly because one has to pay for every byte of data stored over the cloud. Here, we tested efficient management of data generated by an electron cryo-microscopy (cryoEM) lab on a cloud-based environment. The test data was obtained from cryoEM repository EMPIAR. All the images were subjected to an in-house parallelized version of principal component analysis. An efficient cloud-based MapReduce modality was used for parallelization. We showed that large data in order of terabytes could be efficiently reduced to its minimal essential self in a cost-effective scalable manner. Furthermore, on-spot instance on Amazon EC2 was shown to reduce costs by a margin of about 27 percent. This approach could be scaled to data of any large volume and type.

Original language	English
Title of host publication	Proceedings of the 2018 International Conference on Computers in Management and Business, ICCMB 2018
Publisher	ACM
Pages	61-66
Number of pages	6
ISBN (Print)	9781450364232
DOIs	https://doi.org/10.1145/3232174.3232177
Publication status	Published - 25 May 2018
MoE publication type	A4 Article in a conference publication
Event	International Conference on Computers in Management and Business - Oxford, United Kingdom Duration: 25 May 2018 → 27 May 2018

Conference

Conference	International Conference on Computers in Management and Business
Abbreviated title	ICCMB
Country	United Kingdom
City	Oxford
Period	25/05/2018 → 27/05/2018

Keywords

Big Data
Cloud Computing
Cryo-image data
Data reduction
PCA

Access to Document

10.1145/3232174.3232177

Fingerprint Dive into the research topics of 'Exploiting data reduction principles in cloud-based data management for cryo-image data'. Together they form a unique fingerprint.

Cite this

@inproceedings{b37fff7b8a0c47329d1f1268092e5d90,

title = "Exploiting data reduction principles in cloud-based data management for cryo-image data",

abstract = "Cloud computing is a cost-effective way for start-up life sciences laboratories to store and manage their data. However, in many instances the data stored over the cloud could be redundant which makes cloud-based data management inefficient and costly because one has to pay for every byte of data stored over the cloud. Here, we tested efficient management of data generated by an electron cryo-microscopy (cryoEM) lab on a cloud-based environment. The test data was obtained from cryoEM repository EMPIAR. All the images were subjected to an in-house parallelized version of principal component analysis. An efficient cloud-based MapReduce modality was used for parallelization. We showed that large data in order of terabytes could be efficiently reduced to its minimal essential self in a cost-effective scalable manner. Furthermore, on-spot instance on Amazon EC2 was shown to reduce costs by a margin of about 27 percent. This approach could be scaled to data of any large volume and type.",

keywords = "Big Data, Cloud Computing, Cryo-image data, Data reduction, PCA",

author = "Shakil, {Kashish Ara} and Mansaf Alam and Shabih Shakeel and Ari Ora and Samiya Khan",

year = "2018",

month = may,

day = "25",

doi = "10.1145/3232174.3232177",

language = "English",

isbn = "9781450364232",

pages = "61--66",

booktitle = "Proceedings of the 2018 International Conference on Computers in Management and Business, ICCMB 2018",

publisher = "ACM",

note = "International Conference on Computers in Management and Business, ICCMB ; Conference date: 25-05-2018 Through 27-05-2018",

}

Shakil, KA, Alam, M, Shakeel, S, Ora, A & Khan, S 2018, Exploiting data reduction principles in cloud-based data management for cryo-image data. in Proceedings of the 2018 International Conference on Computers in Management and Business, ICCMB 2018. ACM, pp. 61-66, International Conference on Computers in Management and Business, Oxford, United Kingdom, 25/05/2018. https://doi.org/10.1145/3232174.3232177

Exploiting data reduction principles in cloud-based data management for cryo-image data. / Shakil, Kashish Ara; Alam, Mansaf; Shakeel, Shabih; Ora, Ari; Khan, Samiya.

Proceedings of the 2018 International Conference on Computers in Management and Business, ICCMB 2018. ACM, 2018. p. 61-66.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Scientific › peer-review

TY - GEN

T1 - Exploiting data reduction principles in cloud-based data management for cryo-image data

AU - Shakil, Kashish Ara

AU - Alam, Mansaf

AU - Shakeel, Shabih

AU - Ora, Ari

AU - Khan, Samiya

PY - 2018/5/25

Y1 - 2018/5/25

N2 - Cloud computing is a cost-effective way for start-up life sciences laboratories to store and manage their data. However, in many instances the data stored over the cloud could be redundant which makes cloud-based data management inefficient and costly because one has to pay for every byte of data stored over the cloud. Here, we tested efficient management of data generated by an electron cryo-microscopy (cryoEM) lab on a cloud-based environment. The test data was obtained from cryoEM repository EMPIAR. All the images were subjected to an in-house parallelized version of principal component analysis. An efficient cloud-based MapReduce modality was used for parallelization. We showed that large data in order of terabytes could be efficiently reduced to its minimal essential self in a cost-effective scalable manner. Furthermore, on-spot instance on Amazon EC2 was shown to reduce costs by a margin of about 27 percent. This approach could be scaled to data of any large volume and type.

AB - Cloud computing is a cost-effective way for start-up life sciences laboratories to store and manage their data. However, in many instances the data stored over the cloud could be redundant which makes cloud-based data management inefficient and costly because one has to pay for every byte of data stored over the cloud. Here, we tested efficient management of data generated by an electron cryo-microscopy (cryoEM) lab on a cloud-based environment. The test data was obtained from cryoEM repository EMPIAR. All the images were subjected to an in-house parallelized version of principal component analysis. An efficient cloud-based MapReduce modality was used for parallelization. We showed that large data in order of terabytes could be efficiently reduced to its minimal essential self in a cost-effective scalable manner. Furthermore, on-spot instance on Amazon EC2 was shown to reduce costs by a margin of about 27 percent. This approach could be scaled to data of any large volume and type.

KW - Big Data

KW - Cloud Computing

KW - Cryo-image data

KW - Data reduction

KW - PCA

UR - http://www.scopus.com/inward/record.url?scp=85054883269&partnerID=8YFLogxK

UR - https://arxiv.org/abs/1703.10105v1

U2 - 10.1145/3232174.3232177

DO - 10.1145/3232174.3232177

M3 - Conference contribution

AN - SCOPUS:85054883269

SN - 9781450364232

SP - 61

EP - 66

BT - Proceedings of the 2018 International Conference on Computers in Management and Business, ICCMB 2018

PB - ACM

T2 - International Conference on Computers in Management and Business

Y2 - 25 May 2018 through 27 May 2018

ER -