About cookies on this site Our websites require some cookies to function properly (required). In addition, other cookies may be used with your consent to analyze site usage, improve the user experience and for advertising. For more information, please review your options. By visiting our website, you agree to our processing of information as described in IBM’sprivacy statement. To provide a smooth navigation, your cookie preferences will be shared across the IBM web domains listed here.
Publication
IEEE Trans. Inf. Theory
Paper
Memory-Universal Prediction of Stationary Random Processes
Abstract
- We consider the problem of one-step-ahead prediction of a real-valued, stationary, strongly mixing random process {Xi}f:L_00. The best mean-square predictor of Xo is its conditional mean given the entire infinite past {A";}jl}_00. Given a sequence of observations X\ X-z XN, we propose estimators for the conditional mean based on sequences of parametric models of increasing memory and of increasing dimension, for example, neural networks and Legendre polynomials. The proposed estimators select both the model memory and the model dimension, in a data-driven fashion, by minimizing certain complexity regularized least squares criteria. When the underlying predictor function has a finite memory, we establish that the proposed estimators are memory-universal: the proposed estimators, which do not know the true memory, deliver the same statistical performance (rates of integrated mean-squared error) as that delivered by estimators that know the true memory. Furthermore, when the underlying predictor function does not have a finite memory, we establish that the estimator based on Legendre polynomials is consistent. Index Terms-Bernstein inequality, complexity regularization, least-squares loss, Legendre polynomials, Markov processes, memory-universal prediction, mixing processes, model selection, neural networks, time series prediction. © 1998 IEEE.