Sparsity-promoting bootstrap method for large-scale data

Research output: Chapter in Book/Report/Conference proceedingConference contributionScientificpeer-review

1 Citation (Scopus)

Abstract

Performing statistical inference on massive data sets may not be computationally feasible using the conventional statistical inference methodology. In particular, there is a need for methods that are scalable to large volume and variability of data. Moreover, veracity of the inference is crucial. Hence, there is a need to produce quantitative information on the statistical correctness of parameter estimates or decisions. In this paper, we propose a scalable nonparametric bootstrap method that operates with smaller number of distinct data points on multiple disjoint subsets of data. The sampling approach stems from the Bag of Little Bootstraps method and is compatible with distributed storage systems and distributed and parallel processing architectures. Iterative reweighted l1 method is used for each bootstrap replica to find a sparse solution in the face of high-dimensional signal model. The proposed method finds reliable estimates even if the problem is not overdetermined for full data or distinct data subsets by exploiting sparseness. The performance of the proposed method in identifying sparseness in parameter vector and finding confidence intervals and parameter estimates is studied in simulation.

Original languageEnglish
Title of host publicationConference Record of the 50th Asilomar Conference on Signals, Systems and Computers, ACSSC 2016
EditorsMichael B. Matthews
Pages343-348
Number of pages6
ISBN (Electronic)9781538639542
DOIs
Publication statusPublished - 1 Mar 2017
MoE publication typeA4 Article in a conference publication
EventAsilomar Conference on Signals, Systems & Computers - Pasific Grove, United States
Duration: 6 Nov 20169 Nov 2016
Conference number: 50
http://www.asilomarsscconf.org/

Publication series

NameAsilomar Conference on Signals, Systems, and Computers proceedings
PublisherIEEE
ISSN (Electronic)1058-6393

Conference

ConferenceAsilomar Conference on Signals, Systems & Computers
Abbreviated titleASILOMAR
CountryUnited States
CityPasific Grove
Period06/11/201609/11/2016
Internet address

Keywords

  • big data
  • data analysis
  • scalable inference
  • sparse estimation
  • statistical inference

Fingerprint

Dive into the research topics of 'Sparsity-promoting bootstrap method for large-scale data'. Together they form a unique fingerprint.

Cite this