Statistics and Its Interface
Volume 9 (2016)
Special Issue on Statistical and Computational Theory and Methodology for Big Data
Guest Editors: Ming-Hui Chen (University of Connecticut); Radu V. Craiu (University of Toronto); Faming Liang (University of Florida); and Chuanhai Liu (Purdue University)
Direct regression modelling of high-order moments in big data
Pages: 445 – 452
Big data problems present great challenges to statistical analyses, especially from the computational side. In this paper, we consider regression estimation of high-order moments in big data problems based on the U-statistic-based Functional Regression Model (U-FRM) model. The U-FRM model is a nonparametric method that allows direct estimation of higher-order moments without imposing parametric assumptions on the high order-moments. Despite this modeling advantage, its estimation relies on a U-statistics-based estimating equation whose computational complexity is generally too high for big data. In this paper, we propose using the “divide-and-conquer” strategy to construct a computationally more succinct surrogate estimating equation. Through both theoretical proof and simulations, we show that our method significantly reduces the computational time and meanwhile enjoys the same asymptotic behavior as the original estimation method.We then apply our method to a genomic problem to illustrate its performance on real data.
big data, higher-order moment, U-statistics, estimating equation, divide-and-conquer, aggregation, consistency, asymptotic normality, data cube