Sparse principal component regression - Lansangan, Joseph Ryan G. (2013)
Modeling of complex systems is commonly confronted with high dimensional set of independent variables. Similarly, econometric models are usually built using time series data that often exhibit nonstationarity due to the impact of some policies and other economic forces. In both cases, linear regression modeling may yield unstable least squares estimates of the regression coefficients. Principal component regression can provide solution, but in cases where the regressors are nonstationary or the dimension exceeds the sample size, principal components may yield simple averaging of the regressors and the resulting model is difficult to interpret due to biased estimates of the regression coefficients. As a solution, a sparsity constraint is added to the least squares criterion to induce sparsity. The components may then reflect the relative importance of each regressor in a sparse principal component regression (SPCR) model. Simulated and real data are used to illustrate and assess performance of the method. SPCR in many cases leads to better estimation and prediction than conventional principal component regression (PCR). SPCR is able to recognize relative importance of indicators from the sparse components as predictors. In summary, SPCR can be used in modeling high dimensional data, as an intervention strategy in regression with nonstationary time series data, and when there is a general problem of multicollinearity.
Keywords: Regression analysis; Time-series analysis