Statistics: Frisch-Waugh-Lovell Theorem

From MathWiki

Consider a regression of a vector of responses, Y\,\!, on two sets of predictor variables contained in two matrices X_1\,\! and X_2\,\!. The vector of least-squares regression coefficients on X_1\,\! is \hat{\beta}_1 where

Y = X_1 \hat{\beta}_1 + X_2 \hat{\beta}_2 + e with e' [X_1 X_2] = 0 \,

Let Q_2 = I - X_2(X'_2X_2)^{-1}X'_2\,\! be the matrix of the orthogonal projection onto the orthogonal complement of \operatorname{span}(X_2)\,\!.

Then:

Q_2 Y =Q_2 X_1 \hat{\beta}_1 + Q_2 X_2 \hat{\beta}_2 + Q_2 e= Q_2 X_1 \hat{\beta}_1 + 0 + Q_2 e

Now Q_2 e \perp Q_2 X_1 since e'Q'_2Q_2 X_1 = e'Q_2 X_1 =e'(I - P_2)X_1=e'X_1 - e'P_2X_1 =0\,\! since e \perp X_1 and e \perp X_2.

Thus, \hat{\beta}_1 is the regression coefficient of Q_2Y\,\! on Q_2 X_1\,\! .

This is the basis of added-variable plots and an early theorem of Econometrics known as the Frisch-Waugh-Lovell theorem. [Thanks to Barry Smith]


x \perp y