understanding projection matrices Let us look at two linearly independent vectors $v_1$ and $v_2$, which span a plane in $\mathbb{R}^n$ through the origin. Let us form a matrix $V$ out of these vectors so that the first column of $V$ is the vector $v_1$ and the second column is the vector $v_2$. Show that the the matrix $P=V(V^TV)^{-1}V^T$ projects all vectors $x\in\mathbb{R}^n$ orthogonally onto the plane spanned by $v_1$ and $v_2$.

Hint: You need to show two things.


*

*$P\mathbf{x}$ is in the plane, i.e. $P\mathbf{x}\in\text{span}(\mathbf{v}_1,\mathbf{v}_2)$, for all $\mathbf{x}\in\mathbb{R}^n$

*$P\mathbf{x}$ (the projection) and $\mathbf{x}-P\mathbf{x}$ are perpendicular, i.e. $\langle \mathbf{x},\mathbf{x}-P\mathbf{x}\rangle=0$.


To help you get started on (1), the point is that we need to find scalars $c_1$ and $c_2$ so that $\mathbf{x}=c_1\mathbf{v}_1+c_2\mathbf{v}_2$. (That is what it means for $\mathbf{x}$ to lie in the plane spanned by $\mathbf{v}_1$ and $\mathbf{v}_2$.) But $$c_1\mathbf{v}_1+c_2\mathbf{v}_2=V\begin{pmatrix}c_1\\c_2\end{pmatrix}=V\mathbf{c}$$ where $\mathbf{c}$ is the vector $(c_1,c_2)\in\mathbb{R}^2$.
So we want to solve the equation $$V\mathbf{c}=\mathbf{x}$$ for $\mathbf{c}$. Now you have to be a little careful: the matrix $V$ is not square...
