Regularized least square kernel regression for streaming data

We study the use of kernel ridge regression (KRR) in the block-wise streaming data. The algorithm works in an online manner: when a new data block comes in, the algorithm computes a local estimator based on the incoming data block and updates the predictive model by weighted average of all local estimators. Assuming the block data sizes increase at a mild rate and the regularization parameters are selected adaptively according to the sample size of all available data at the time of updating the model, we prove the convergence of the average KRR estimator. The rate is optimal when the regression function can be well approximated by the reproducing kernel Hilbert space in the $L^2$ sense.

Keywords

learning theory, kernel ridge regression, streaming data, online learning, adaptive underregularization

2010 Mathematics Subject Classification

68Q32, 68T05, 68W27

Full Text (PDF format)

The work by Hongwei Sun described in this paper is supported by National Natural Science Foundation of China (Grants No. 11671171 and 11871167).

The work by Qiang Wu is partially supported by the Simons Foundation Collaboration Grant (Award ID 712916).

The three authors contributed equally to this paper.

Received 14 July 2020

Accepted 27 January 2021

Published 2 August 2021