Statistics and Its Interface
Volume 8 (2015)
Special Issue on Modern Bayesian Statistics (Part II)
Guest Editor: Ming-Hui Chen (University of Connecticut)
Difficulty of selecting among multilevel models using predictive accuracy
Pages: 153 – 160
As a simple and compelling approach for estimating out-of-sample prediction error, cross-validation naturally lends itself to the task of model comparison. However, even with moderate sample size, it can be surprisingly difficult to compare multilevel models based on predictive accuracy. Using a hierarchical model fit to large survey data with a battery of questions, we demonstrate that even though cross-validation might give good estimates of pointwise out-of-sample prediction error, it is not always a sensitive instrument for model comparison.
multilevel models, predictive accuracy, model selection, sample survey, cross-validation
2010 Mathematics Subject Classification
Primary 62F15. Secondary 62D05.