Scalable Statistical Inference Using Distributed Bootstrapping and Iterative ℓ ₁ -Norm Minimization

In this paper we address the problem of performing statistical inference for large-scale data sets where volume and dimensionality of the data may be so high that it cannot be processed or stored in a single unit. In particular, we focus on bootstrapping based methods that can provide quantitative information on the accuracy of the inference such as confidence intervals without explicit assumptions on the probability models. We propose a scalable distributed boot- strap method that uses iterative estimation equations favoring sparse solution. Scalability is achieved by applying bootstrapping to multiple smaller distinct subsets generated by resampling the full data without replacement similarly to BLB method [1]. An iteratively reweighted ell-{1}-norm minimizing estimation equations are applied to each bootstrap sample. Such estimators allow for parameter estimation and inference even for moderately underdetermined systems as well as performing variable selection by promoting a sparse parameter vector. Estimation problems may become underdetermined for the distinct subsets of data even if the full large scale problem would be overdetermined. The performance of the presented approach is studied in extensive simulations. It is demonstrated that the method gives smaller Root MSE and significantly lower bias than bootstrap employing widely used sparse estimator BPDN. Moreover, better performance is obtained in variable selection in terms of classification error rate (CER) and recovery rate (RER) in identifying sparse parameters. Estimated confidence intervals are also highly concentrated about the true parameter values.