Statistics: Added variable plot

From MathWiki

In a multiple regression, the added variable plot for a predictor X, say, is the plot showing the residual of Y against all predictors except X against the residual of X on all predictors except X, of course.

One can think of the added variable plot as a particular view of higher dimensional data. The added variable plot views down the intersection of the plane of the regression of Y on all predictors and the plane of the regression of Y on all predictors except X. The plane of the regression of X on all predictors except X also intersects in the same line.

The graphs below show regressions of Health on Weight, Health on Height and Weight and, in the plane of predictor variables, the regression of Height on Weight. The second graph shows the data rotated so one is looking down the intersection of the regression planes. The horizontal displacement in this plot is proportional to the residual of Height on Weight. The residuals of the regression of Health on Weight can be viewed above and below the blue regression plane which is viewed on edge.

Front view of regression planes

Image:AddedVariablePlotFrontView.png

Added variable plot view

Note that the horizontal displacement is proportional to the residual of Height on Weight. In the added variable plot, the blue line would be the horizontal axis. The red line is the least-squares line though the added variable plot and also happens to be the edge of the multiple regression plane. Thus the added variable plot is a way of looking at the data that turns three regression planes into lines:

  • the regression plane of Height on Weight becomes the vertical axis,
  • the regression plane of Health on Weight becomes the horizontal axis,
  • the regression plane of Health on both Height and Weight becomes the least-squares regression line

Image:AddedVariablePlotAVPView.png

Frisch-Waugh-Lovell Theorem

The fact that a simple regression using the added variable plot produces results for the targetted variable that are essentially equivalent to those of a multiple regression is known in econometrics as the Frisch-Waugh-Lovell Theorem (http://en.wikipedia.org/wiki/Frisch%E2%80%93Waugh%E2%80%93Lovell_theorem)