Question: 13. Exercises 12–14 concern an \(m \times n\) matrix \(A\) with a reduced singular value decomposition, \(A = {U_r}D{V_r}^T\), and the pseudoinverse \({A^ + } = {U_r}{D^{ - 1}}{V_r}^T\).

Suppose the equation\(A{\rm{x}} = {\rm{b}}\)is consistent, and let\({{\rm{x}}^ + } = {A^ + }{\rm{b}}\). By Exercise 23 in Section 6.3, there is exactly one vector\({\rm{p}}\)in Row\(A\)such that\(A{\rm{p}} = {\rm{b}}\). The following steps prove that\({{\rm{x}}^ + } = {\rm{p}}\)and\({{\rm{x}}^ + }\)is the minimum length solution of\(A{\rm{x}} = {\rm{b}}\).

1. Show that \({{\rm{x}}^ + }\) is in Row \(A\). (Hint: Write \({\rm{b}}\) as \(A{\rm{x}}\) for some \({\rm{x}}\), and use Exercise 12.)
2. Show that\({{\rm{x}}^ + }\)is a solution of\(A{\rm{x}} = {\rm{b}}\).
3. Show that if \({\rm{u}}\) is any solution of \(A{\rm{x}} = {\rm{b}}\), then \(\left\| {{{\rm{x}}^ + }} \right\| \le \left\| {\rm{u}} \right\|\), with equality only if \({\rm{u}} = {{\rm{x}}^ + }\).

It is given that the system\(A{\rm{x}} = {\rm{b}}\)is consistent and\({{\rm{x}}^ + } = {A^ + }{\rm{b}}\), then the system\(A{\rm{x}} = {\rm{b}}\)can be simplified as follows:

\(\begin{array}{c}{{\rm{x}}^ + } = {A^ + }{\rm{b}}\\ = {A^ + }A{\rm{x}}\end{array}\)

As \({{\rm{x}}^ + } = {A^ + }A{\rm{x}}\), so\({{\rm{x}}^ + }\) is the orthogonal projection of \({\rm{x}}\) onto Row \(A\).

Substitute\({{\rm{x}}^ + } = {A^ + }A{\rm{x}}\), into\(A{{\rm{x}}^ + }\), and simplify using\(A{\rm{x}} = {\rm{b}}\), as follows:

\(\begin{array}{c}A{{\rm{x}}^ + } = A\left( {{A^ + }A{\rm{x}}} \right)\\ = A{A^ + }A{\rm{x}}\\ = A{\rm{x}}\\ = {\rm{b}}\end{array}\)

Thus, it is shown that \({{\rm{x}}^ + }\) is the solution of \(A{\rm{x}} = {\rm{b}}\).

Suppose the system\(A{\rm{x}} = {\rm{b}}\)issatisfied by another basis\({\rm{u}}\), such that,\(A{\rm{u}} = {\rm{b}}\). It is also known that\({{\rm{x}}^ + }\)is the orthogonal projection of \({\rm{x}}\) onto Row \(A\).

Apply the Pythagorean theorem on\({\left\| {\rm{u}} \right\|^2}\), as follows:

\(\begin{array}{c}{\left\| {\rm{u}} \right\|^2} = {\left\| {{{\rm{x}}^ + }} \right\|^2} + {\left\| {{\rm{u}} - {{\rm{x}}^ + }} \right\|^2}\\ \ge {\left\| {{{\rm{x}}^ + }} \right\|^2}\end{array}\)

So \(\left\| {{{\rm{x}}^ + }} \right\| < \left\| {\rm{u}} \right\|\), and equality holds if \({\bf{u}} = {{\rm{x}}^ + }\).