Question: 12. Exercises 12–14 concern an \(m \times n\) matrix \(A\) with a reduced singular value decomposition, \(A = {U_r}D{V_r}^T\), and the pseudoinverse \({A^ + } = {U_r}{D^{ - 1}}{V_r}^T\).

Verify the properties of\({A^ + }\):

a. For each\({\rm{y}}\)in\({\mathbb{R}^m}\),\(A{A^ + }{\rm{y}}\)is the orthogonal projection of\({\rm{y}}\)onto\({\rm{Col}}\,A\).

b. For each\({\rm{x}}\)in\({\mathbb{R}^n}\),\({A^ + }A{\rm{x}}\)is the orthogonal projection of\({\rm{x}}\)onto\({\rm{Row}}\,A\).

c. \(A{A^ + }A = A\)and \({A^ + }A{A^ + } = {A^ + }\).

As the matrix\({V_r}\)has orthonormal columns, the matrix\(A{A^ + }{\rm{y}}\)can be written in terms of\({U_r}\),as shown below:

\(\begin{array}{c}A{A^ + }{\rm{y}} = \left( {{U_r}D{V_r}^T} \right)\left( {{V_r}{D^{ - 1}}{U_r}^T} \right){\rm{y}}\\ = \left( {{U_r}D{D^{ - 1}}{U_r}^T} \right){\rm{y}}\\ = \left( {{U_r}{U_r}^T} \right){\rm{y}}\end{array}\)

According to theorem 10, the matrix \(\left( {{U_r}{U_r}^T} \right){\rm{y}}\) is the orthogonal projection of \({\rm{y}}\) onto \({\rm{Col}}\,\,{U_r}\), which is equal to \({\rm{Col}}\,\,A\). Thus, \(A{A^ + }{\rm{y}}\) is the orthogonal projection of \({\rm{y}}\) onto \({\rm{Col}}\,\,A\).

As the matrix\({U_r}\)has orthonormal columns, the matrix\(A{A^ + }{\rm{x}}\)can be written in terms of\({V_r}\),as shown below:

\(\begin{array}{c}A{A^ + }{\rm{x}} = \left( {{V_r}D{U_r}^T} \right)\left( {{U_r}{D^{ - 1}}{V_r}^T} \right){\rm{y}}\\ = \left( {{V_r}D{D^{ - 1}}{V_r}^T} \right){\rm{y}}\\ = \left( {{V_r}{V_r}^T} \right){\rm{y}}\end{array}\)

According to theorem 10, the matrix \(\left( {{V_r}{V_r}^T} \right){\rm{x}}\) is the orthogonal projection of \({\rm{x}}\) onto \({\rm{Col}}\,\,{V_r}\), which is equal to \({\rm{Row}}\,A\). Thus, \(A{A^ + }{\rm{y}}\) is the orthogonal projection of \({\rm{x}}\) onto \({\rm{Row}}\,A\).

Apply the reduced singular value decomposition, definition of\({A^ + }\), and the associativity of matrix multiplication to simplify\(A{A^ + }A\)and\({A^ + }A{A^ + }\), as shown below:

\(\begin{array}{c}A{A^ + }A = \left( {{U_r}D{V_r}^T} \right)\left( {{V_r}{D^{ - 1}}{U_r}^T} \right)\left( {{U_r}D{V_r}^T} \right)\\ = \left( {{U_r}D{D^{ - 1}}{U_r}^T} \right)\left( {{U_r}D{V_r}^T} \right)\\ = {U_r}D{D^{ - 1}}{V_r}^T\\ = {U_r}D{V_r}^T\\ = A\end{array}\)

And,

\(\begin{array}{c}{A^ + }A{A^ + } = \left( {{V_r}{D^{ - 1}}{U_r}^T} \right)\left( {{U_r}D{V_r}^T} \right)\left( {{V_r}{D^{ - 1}}{U_r}^T} \right)\\ = \left( {{V_r}{D^{ - 1}}D{U_r}^T} \right)\left( {{V_r}D{U_r}^T} \right)\\ = {V_r}D{D^{ - 1}}{U_r}^T\\ = {V_r}D{U_r}^T\\ = {A^ + }\end{array}\)

Thus, it is verified that \(A{A^ + }A = A\) and \({A^ + }A{A^ + } = {A^ + }\).