Statistics and Its Interface

Volume 2 (2009)

Number 3

FIRST: Combining forward iterative selection and shrinkage in high dimensional sparse linear regression

Pages: 341 – 348

DOI: https://dx.doi.org/10.4310/SII.2009.v2.n3.a7

Authors

Subhashis Ghosal (Department of Statistics, North Carolina State University, Raleigh, N.C., U.S.A.)

Wook Yeon Hwang (Department of Statistics, North Carolina State University, Raleigh, N.C., U.S.A.)

Hao Helen Zhang (Department of Statistics, North Carolina State University, Raleigh, N.C., U.S.A.)

Abstract

We propose a new class of variable selection techniques for regression in high dimensional linear models based on a forward selection version of the LASSO, adaptive LASSO or elastic net, respectively to be called as forward iterative regression and shrinkage technique (FIRST), adaptive FIRST and elastic FIRST. These methods seem to work effectively for extremely sparse high dimensional linear models. We exploit the fact that the LASSO, adaptive LASSO and elastic net have closed form solutions when the predictor is onedimensional. The explicit formula is then repeatedly used in an iterative fashion to build the model until convergence occurs. By carefully considering the relationship between estimators at successive stages, we develop fast algorithms to compute our estimators. The performance of our new estimators are compared with commonly used estimators in terms of predictive accuracy and errors in variable selection.

Keywords

LASSO, elastic net, high dimension, sparsity, variable selection

2010 Mathematics Subject Classification

Primary 62J05. Secondary 62J07.

Full Text (PDF format)

Published 1 January 2009