In Exercises 9-12 find (a) the orthogonal projection of b onto \({\bf{Col}}A\) and (b) a least-squares solution of \(A{\bf{x}} = {\bf{b}}\).

12. \(A = \left[ {\begin{array}{{}{}}{\bf{1}}&{\bf{1}}&{\bf{0}}\\{\bf{1}}&{\bf{0}}&{ - {\bf{1}}}\\{\bf{0}}&{\bf{1}}&{\bf{1}}\\{ - {\bf{1}}}&{\bf{1}}&{ - {\bf{1}}}\end{array}} \right]\), \({\bf{b}} = \left( {\begin{array}{{}{}}{\bf{2}}\\{\bf{5}}\\{\bf{6}}\\{\bf{6}}\end{array}} \right)\)

The orthogonal projection of b onto \({\rm{Col}}A\) is:

\(\begin{aligned}{}{\bf{\hat b}} &= {\rm{pro}}{{\rm{j}}_{{\rm{col}}A}}{\bf{b}}\\ &= \frac{{{\bf{b}} \cdot {{\bf{v}}_1}}}{{{{\bf{v}}_1} \cdot {{\bf{v}}_1}}}{{\bf{v}}_1} + \frac{{{\bf{b}} \cdot {{\bf{v}}_2}}}{{{{\bf{v}}_2} \cdot {{\bf{v}}_2}}}{{\bf{v}}_2} + \frac{{{\bf{b}} \cdot {{\bf{v}}_3}}}{{{{\bf{v}}_3} \cdot {{\bf{v}}_3}}}{{\bf{v}}_3}\\ &= \frac{{\left( {\begin{aligned}{{}{}}2&5&6&6\end{aligned}} \right)\left( {\begin{aligned}{{}{}}1\\1\\0\\{ - 1}\end{aligned}} \right)}}{{\left( {\begin{aligned}{{}{}}1&1&0&{ - 1}\end{aligned}} \right)\left( {\begin{aligned}{{}{}}1\\1\\0\\{ - 1}\end{aligned}} \right)}}{{\bf{v}}_1} + \frac{{\left( {\begin{aligned}{{}{}}2&5&6&6\end{aligned}} \right)\left( {\begin{aligned}{{}{}}1\\0\\1\\1\end{aligned}} \right)}}{{\left( {\begin{aligned}{{}{}}1&0&1&1\end{aligned}} \right)\left( {\begin{aligned}{{}{}}1\\0\\1\\1\end{aligned}} \right)}}{{\bf{v}}_2} + \frac{{\left( {\begin{aligned}{{}{}}2&5&6&6\end{aligned}} \right)\left( {\begin{aligned}{{}{}}0\\{ - 1}\\1\\{ - 1}\end{aligned}} \right)}}{{\left( {\begin{aligned}{{}{}}0&{ - 1}&1&{ - 1}\end{aligned}} \right)\left( {\begin{aligned}{{}{}}0\\{ - 1}\\1\\{ - 1}\end{aligned}} \right)}}{{\bf{v}}_3}\end{aligned}\)

Solve further,

\(\begin{aligned}{}{\bf{\hat b}} &= \frac{1}{3}\left( {\begin{aligned}{{}{}}1\\1\\0\\{ - 1}\end{aligned}} \right) + \frac{{14}}{3}\left( {\begin{aligned}{{}{}}1\\0\\1\\1\end{aligned}} \right) + \frac{{\left( { - 5} \right)}}{3}\left( {\begin{aligned}{{}{}}0\\{ - 1}\\1\\{ - 1}\end{aligned}} \right)\\ &= \left( {\begin{aligned}{{}{}}5\\2\\3\\6\end{aligned}} \right)\end{aligned}\)

The orthogonal projection of b onto ColA is \(\left( {\begin{aligned}{{}{}}5\\2\\3\\6\end{aligned}} \right)\).

Find the product \({A^T}A\).

\(\begin{aligned}{}{A^T}A &= \left( {\begin{aligned}{{}{}}1&1&0&{ - 1}\\1&0&1&1\\0&{ - 1}&1&{ - 1}\end{aligned}} \right)\left( {\begin{aligned}{{}{}}1&1&0\\1&0&{ - 1}\\0&1&1\\{ - 1}&1&{ - 1}\end{aligned}} \right)\\ &= \left( {\begin{aligned}{{}{}}3&0&0\\0&3&0\\0&0&3\end{aligned}} \right)\end{aligned}\)

Find the product \({A^T}{\bf{b}}\).

\(\begin{aligned}{}{A^T}{\bf{b}} &= \left( {\begin{aligned}{{}{}}1&1&0&{ - 1}\\1&0&1&1\\0&{ - 1}&1&{ - 1}\end{aligned}} \right)\left( {\begin{aligned}{{}{}}2\\5\\6\\6\end{aligned}} \right)\\ &= \left( {\begin{aligned}{{}{}}1\\{14}\\{ - 5}\end{aligned}} \right)\end{aligned}\)

The normal equation can be written as:

\(\begin{aligned}{}\left( {{A^T}A} \right){\bf{x}} &= {A^T}{\bf{b}}\\\left( {\begin{aligned}{{}{}}3&0&0\\0&3&0\\0&0&3\end{aligned}} \right)\left( {\begin{aligned}{{}{}}{{x_1}}\\{{x_2}}\\{{x_3}}\end{aligned}} \right) &= \left( {\begin{aligned}{{}{}}1\\{14}\\{ - 5}\end{aligned}} \right)\end{aligned}\)

The least-square solution can be calculated as follows:

\(\begin{aligned}{}{\bf{\hat x}} &= {\left( {{A^T}A} \right)^{ - 1}}{A^T}{\bf{b}}\\& = {\left( {\begin{aligned}{{}{}}3&0&0\\0&3&0\\0&0&3\end{aligned}} \right)^{ - 1}}\left( {\begin{aligned}{{}{}}1\\{14}\\{ - 5}\end{aligned}} \right)\\ &= \left( {\begin{aligned}{{}{}}{\frac{1}{3}}&0&0\\0&{\frac{1}{3}}&0\\0&0&{\frac{1}{3}}\end{aligned}} \right)\left( {\begin{aligned}{{}{}}1\\{14}\\{ - 5}\end{aligned}} \right)\\ &= \left( {\begin{aligned}{{}{}}{\frac{1}{3}}\\{\frac{{14}}{3}}\\{ - \frac{5}{3}}\end{aligned}} \right)\end{aligned}\)

Thus, the least square solution is \(\left( {\begin{aligned}{{}{}}{\frac{1}{3}}\\{\frac{{14}}{3}}\\{ - \frac{5}{3}}\end{aligned}} \right)\).