Robust, Sparse and Scalable Inference Using Bootstrap and Variable Selection Fusion

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

Abstract

In this paper, we address the challenging problem of conducting statistical inference for large-scale data sets in the presence of sparsity and outlying observations. In particular, processing and storing such data on a single computing node may be infeasible due to its high volume and dimensionality. Therefore, the large-scale data is subdivided into smaller distinct subsets that may be stored and processed in different nodes. We propose a robust and scalable statistical inference method using a two-stage algorithm where variable selection is performed via fusing the selected support from each distinct subset of data. The actual parameter and confidence interval estimation takes place in the second stage using a robust extension of Bag of Little Bootstraps (BLB) technique. In order to exploit sparsity and ensure robustness, MM-Lasso estimator is used to select variables for each subset of data. The selections are then fused to find the support for the original large-scale data. In the second stage, the robust MM-estimator is used for the selected support. The simulation studies demonstrated the highly reliable performance of the algorithm in variable selection and providing reliable confidence intervals even if the estimation problem in the subsets is slightly under-determined.

Original languageEnglish
Title of host publication2019 IEEE 8th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, CAMSAP 2019 - Proceedings
PublisherIEEE
Pages271-275
Number of pages5
ISBN (Electronic)9781728155494
DOIs
Publication statusPublished - 1 Dec 2019
MoE publication typeA4 Article in a conference publication
EventIEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing - Guadeloupe, Le Gosier, Guadeloupe
Duration: 15 Dec 201918 Dec 2019
https://camsap19.ig.umons.ac.be

Workshop

WorkshopIEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing
Abbreviated titleCAMSAP
CountryGuadeloupe
CityLe Gosier
Period15/12/201918/12/2019
Internet address

Keywords

  • bootstrap
  • high-dimensional
  • large-scale
  • robust
  • sparsity
  • statistical inference

Fingerprint Dive into the research topics of 'Robust, Sparse and Scalable Inference Using Bootstrap and Variable Selection Fusion'. Together they form a unique fingerprint.

Cite this