Projects per year
In this paper, we address the challenging problem of conducting statistical inference for large-scale data sets in the presence of sparsity and outlying observations. In particular, processing and storing such data on a single computing node may be infeasible due to its high volume and dimensionality. Therefore, the large-scale data is subdivided into smaller distinct subsets that may be stored and processed in different nodes. We propose a robust and scalable statistical inference method using a two-stage algorithm where variable selection is performed via fusing the selected support from each distinct subset of data. The actual parameter and confidence interval estimation takes place in the second stage using a robust extension of Bag of Little Bootstraps (BLB) technique. In order to exploit sparsity and ensure robustness, MM-Lasso estimator is used to select variables for each subset of data. The selections are then fused to find the support for the original large-scale data. In the second stage, the robust MM-estimator is used for the selected support. The simulation studies demonstrated the highly reliable performance of the algorithm in variable selection and providing reliable confidence intervals even if the estimation problem in the subsets is slightly under-determined.
|Title of host publication||2019 IEEE 8th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, CAMSAP 2019 - Proceedings|
|Number of pages||5|
|Publication status||Published - 1 Dec 2019|
|MoE publication type||A4 Article in a conference publication|
|Event||IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing - Guadeloupe, Le Gosier, Guadeloupe|
Duration: 15 Dec 2019 → 18 Dec 2019
|Workshop||IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing|
|Period||15/12/2019 → 18/12/2019|
- statistical inference
FingerprintDive into the research topics of 'Robust, Sparse and Scalable Inference Using Bootstrap and Variable Selection Fusion'. Together they form a unique fingerprint.
- 1 Finished
01/09/2015 → 31/08/2019
Project: Academy of Finland: Other research funding