Let B be an \(n \times n\) symmetric matrix such that \({B^{\bf{2}}} = B\). Any such matrix is called a projection matrix (or an orthogonal projection matrix.) Given any y in \({\mathbb{R}^n}\), let \({\bf{\hat y}} = B{\bf{y}}\)and\({\bf{z}} = {\bf{y}} - {\bf{\hat y}}\).

a) Show that z is orthogonal to \({\bf{\hat y}}\).

b) Let W be the column space of B. Show that y is the sum of a vector in W and a vector in \({W^ \bot }\). Why does this prove that By is the orthogonal projection of y onto the column space of B?

Find the product of z and \({\bf{\hat y}}\).

\(\begin{aligned}{}{\bf{z\hat y}} &= \left( {{\bf{y}} - {\bf{\hat y}}} \right)\left( {B{\bf{y}}} \right)\\ & = {\bf{y}}\left( {B{\bf{y}}} \right) - \left( {B{\bf{y}}} \right)\left( {B{\bf{y}}} \right)\\ & = {\bf{y}}\left( {B{\bf{y}}} \right) - B\left( {{\bf{y}}B} \right){\bf{y}}\\ & = {\bf{y}}\left( {B{\bf{y}}} \right) - {\bf{y}}\left( {BB} \right){\bf{y}}\\ & = {\bf{y}}\left( {B{\bf{y}}} \right) - {\bf{y}}\left( {B{\bf{y}}} \right)\\& = 0\end{aligned}\)

Thus, z is orthogonal to \({\bf{\hat y}}\).

As W is a column space of B, therefore;

\(\begin{aligned}{}\left( {{\bf{y}} - {\bf{\hat y}}} \right) \cdot \left( {B{\bf{u}}} \right) & = \left[ {B\left( {{\bf{y}} - {\bf{\hat y}}} \right)} \right] \cdot {\bf{u}}\\ & = \left[ {B{\bf{y}} - BB{\bf{y}}} \right] \cdot {\bf{u}}\\ & = \left[ {B{\bf{y}} - B{\bf{y}}} \right] \cdot {\bf{u}}\\ & = 0\end{aligned}\)

Thus, \({\bf{y}} - {\bf{\hat y}}\) is in \({W^ \bot }\), and decomposition \({\bf{y}} = {\bf{\hat y}} + \left( {{\bf{y}} - {\bf{\hat y}}} \right)\) express y as the sum of vector in W and another vector in \({W^ \bot }\).

According to Orthogonal Decomposition Theorem, the decomposition \({\bf{y}} = {\bf{\hat y}} + \left( {{\bf{y}} - {\bf{\hat y}}} \right)\) is unique.Thus, \({\bf{\hat y}}\) must be the projection of y.