Doubly Robust Feature Selection with Mean and Variance Outlier Detection and Oracle Properties
Abstract
We propose a general approach to handle data contaminations that might disrupt the performance of feature selection and estimation procedures for high-dimensional linear models. Specifically, we consider the co-occurrence of mean-shift and variance-inflation outliers, which can be modeled as additional fixed and random components, respectively, and evaluated independently. Our proposal performs feature selection while detecting and down-weighting variance-inflation outliers, detecting and excluding mean-shift outliers, and retaining non-outlying cases with full weights. Feature selection and mean-shift outlier detection are performed through a robust class of nonconcave penalization methods. Variance-inflation outlier detection is based on the penalization of the restricted posterior mode. The resulting approach satisfies a robust oracle property for feature selection in the presence of data contamination – which allows the number of features to exponentially increase with the sample size – and detects truly outlying cases of each type with asymptotic probability one. This provides an optimal trade-off between a high breakdown point and efficiency. Computationally efficient heuristic procedures are also presented. We illustrate the finite-sample performance of our proposal through an extensive simulation study and a real-world application.
Under review.
Luca Insolia
Postdoctoral Researcher
My primary research interests concern robust statistics and high-dimensional modeling. During my PhD, I developed statistical methodologies for analyzing sparse regression problems affected by different forms of adversarial data contamination. The developed methodologies encompass continuous optimization methods as well as mixed-integer programming techniques. I applied these tools to analyze biomedical data and to investigate the main possible drivers of honey bee colony loss.
Francesca Chiaromonte
Full Professor
Marco Riani
Full professor
