Question:Examine powers of a regular stochastic matrix.

a. Compute\({p^k}\)for\(k = 2,3,4,5\), when

\(P = \left[ {\begin{array}{*{20}{c}}{.3355}&{.3682}&{.3067}&{.0389}\\{.2663}&{.2723}&{.3277}&{.5451}\\{.1935}&{.1502}&{.1589}&{.2395}\\{.2047}&{.2093}&{.2067}&{.1765}\end{array}} \right]\)

Display calculations to four decimal places. What happens to the columns of\({p^k}\)as\(k\)increases? Compute thesteady-state vector for P.

b. Compute\({Q^k}\)for\(k = 10,20,...,80\), when

\(Q = \left[ {\begin{array}{*{20}{c}}{.97}&{.05}&{.10}\\0&{.90}&{.05}\\{.03}&{.05}&{.85}\end{array}} \right]\)

(Stability for\({Q^k}\)to four decimal places may require\(k = 116\)or more.) Compute the steady-state vector for Q.

c. Use Theorem 18 to explain what you found in parts (a) and (b).

Compute the powers of the stochastic matrix using the MATLAB command shown below:

Substitute 2 for\(k\)in\({p^k}\)to obtain\({p^2}\).

\(\begin{array}{l} > > P = \left[ \begin{array}{l}{\rm{.3355 }}{\rm{.3682 }}{\rm{.3067 }}{\rm{.0389; }}{\rm{.2663 }}{\rm{.2723 }}{\rm{.3067 }}{\rm{.5451;}}\\{\rm{.1935 }}{\rm{.1502 }}{\rm{.1589 }}{\rm{.2395; }}{\rm{.2047 }}{\rm{.2093 }}{\rm{.2067 }}{\rm{.1765}}\end{array} \right];\\ > > A = P^2\end{array}\)

\({P^2} = \left[ {\begin{array}{*{20}{c}}{.2779}&{.2780}&{.2803}&{.2941}\\{.3368}&{.3355}&{.3357}&{.3335}\\{.1847}&{.1861}&{.1833}&{.1697}\\{.2005}&{.2004}&{.2007}&{.2027}\end{array}} \right]\)

Substitute 3 for\(k\)in\({P^k}\)to obtain\({P^3}\).

\(\begin{array}{l} > > P = \left[ \begin{array}{l}{\rm{.3355 }}{\rm{.3682 }}{\rm{.3067 }}{\rm{.0389; }}{\rm{.2663 }}{\rm{.2723 }}{\rm{.3067 }}{\rm{.5451;}}\\{\rm{.1935 }}{\rm{.1502 }}{\rm{.1589 }}{\rm{.2395; }}{\rm{.2047 }}{\rm{.2093 }}{\rm{.2067 }}{\rm{.1765}}\end{array} \right];\\ > > A = P^3\end{array}\)

\({P^3} = \left[ {\begin{array}{*{20}{c}}{.2817}&{.2817}&{.2817}&{.2814}\\{.3356}&{.3356}&{.3355}&{.3352}\\{.1817}&{.1817}&{.1819}&{.1825}\\{.2010}&{.2010}&{.2010}&{.2009}\end{array}} \right]\)

Substitute 4 for\(k\)in\({P^k}\)to obtain\({P^4}\).

\(\begin{array}{l} > > P = \left[ \begin{array}{l}{\rm{.3355 }}{\rm{.3682 }}{\rm{.3067 }}{\rm{.0389; }}{\rm{.2663 }}{\rm{.2723 }}{\rm{.3067 }}{\rm{.5451;}}\\{\rm{.1935 }}{\rm{.1502 }}{\rm{.1589 }}{\rm{.2395; }}{\rm{.2047 }}{\rm{.2093 }}{\rm{.2067 }}{\rm{.1765}}\end{array} \right];\\ > > A = P^4\end{array}\)

\({P^4} = \left[ {\begin{array}{*{20}{c}}{.2816}&{.2816}&{.2816}&{.2816}\\{.3355}&{.3355}&{.3355}&{.3355}\\{.1819}&{.1819}&{.1819}&{.1819}\\{.2009}&{.2009}&{.2009}&{.2009}\end{array}} \right]\)

Substitute 5 for\(k\)in\({P^k}\)to obtain\({P^5}\).

\(\begin{array}{l} > > P = \left[ \begin{array}{l}{\rm{.3355 }}{\rm{.3682 }}{\rm{.3067 }}{\rm{.0389; }}{\rm{.2663 }}{\rm{.2723 }}{\rm{.3067 }}{\rm{.5451;}}\\{\rm{.1935 }}{\rm{.1502 }}{\rm{.1589 }}{\rm{.2395; }}{\rm{.2047 }}{\rm{.2093 }}{\rm{.2067 }}{\rm{.1765}}\end{array} \right];\\ > > A = P^5\end{array}\)

\({P^5} = \left[ {\begin{array}{*{20}{c}}{.2816}&{.2816}&{.2816}&{.2816}\\{.3355}&{.3355}&{.3355}&{.3355}\\{.1819}&{.1819}&{.1819}&{.1819}\\{.2009}&{.2009}&{.2009}&{.2009}\end{array}} \right]\)

Thus, as\(k\)increases,\({p^k}\)converges to the common vector\(\left[ {\begin{array}{*{20}{c}}{.2816}\\{.3355}\\{.1819}\\{.2009}\end{array}} \right]\).

It means the steady-state vector q is \(\left[ {\begin{array}{*{20}{c}}{.2816}\\{.3355}\\{.1819}\\{.2009}\end{array}} \right]\).

Compute the powers of the stochastic matrix using the MATLAB command shown below:

Substitute 10 for\(k\)in\({Q^k}\)to obtain\({Q^{10}}\).

\(\begin{array}{l} > > Q = \left[ {{\rm{0}}{\rm{.97 }}{\rm{.0}}{\rm{.05 0}}{\rm{.10; 0 0}}{\rm{.90 0}}{\rm{.05; 0}}{\rm{.03 0}}{\rm{.05 0}}{\rm{.85}}} \right];\\ > > A = Q^10\end{array}\)

\({Q^{10}} = \left[ {\begin{array}{*{20}{c}}{.8222}&{.4044}&{.5385}\\{.0324}&{.3966}&{.1666}\\{.1453}&{.1990}&{.2949}\end{array}} \right]\)

Substitute 20 for\(k\)in\({Q^k}\)to obtain\({Q^{20}}\).

\(\begin{array}{l} > > Q = \left[ {{\rm{0}}{\rm{.97 }}{\rm{.0}}{\rm{.05 0}}{\rm{.10; 0 0}}{\rm{.90 0}}{\rm{.05; 0}}{\rm{.03 0}}{\rm{.05 0}}{\rm{.85}}} \right];\\ > > A = Q^20\end{array}\)

\({Q^{20}} = \left[ {\begin{array}{*{20}{c}}{.7674}&{.6000}&{.6690}\\{.0637}&{.2036}&{.1326}\\{.1688}&{.1964}&{.1984}\end{array}} \right]\)

Substitute 30 for\(k\)in\({Q^k}\)to obtain\({Q^{30}}\).

\(\begin{array}{l} > > Q = \left[ {{\rm{0}}{\rm{.97 }}{\rm{.0}}{\rm{.05 0}}{\rm{.10; 0 0}}{\rm{.90 0}}{\rm{.05; 0}}{\rm{.03 0}}{\rm{.05 0}}{\rm{.85}}} \right];\\ > > A = Q^30\end{array}\)

\({Q^{30}} = \left[ {\begin{array}{*{20}{c}}{.7477}&{.6815}&{.7105}\\{.0783}&{.1329}&{.1074}\\{.17}&{}&{}\end{array}} \right]\)

Substitute 40 for\(k\)in\({Q^k}\)to obtain\({Q^{40}}\).

\(\begin{array}{l} > > Q = \left[ {{\rm{0}}{\rm{.97 }}{\rm{.0}}{\rm{.05 0}}{\rm{.10; 0 0}}{\rm{.90 0}}{\rm{.05; 0}}{\rm{.03 0}}{\rm{.05 0}}{\rm{.85}}} \right];\\ > > A = Q^40\end{array}\)

\({Q^{40}} = \left[ {\begin{array}{*{20}{c}}{.7401}&{.7140}&{.7257}\\{.0843}&{.1057}&{.0960}\\{.1756}&{.1802}&{.1783}\end{array}} \right]\)

Substitute 50 for\(k\)in\({Q^k}\)to obtain\({Q^{50}}\).

\(\begin{array}{l} > > Q = \left[ {{\rm{0}}{\rm{.97 }}{\rm{.0}}{\rm{.05 0}}{\rm{.10; 0 0}}{\rm{.90 0}}{\rm{.05; 0}}{\rm{.03 0}}{\rm{.05 0}}{\rm{.85}}} \right];\\ > > A = Q^50\end{array}\)

\({Q^{50}} = \left[ {\begin{array}{*{20}{c}}{.7372}&{.7269}&{.7315}\\{.0867}&{.0951}&{.0913}\\{.1761}&{.1780}&{.1772}\end{array}} \right]\)

Substitute 60 for\(k\)in\({Q^k}\)to obtain\({Q^{60}}\).

\(\begin{array}{l} > > Q = \left[ {{\rm{0}}{\rm{.97 }}{\rm{.0}}{\rm{.05 0}}{\rm{.10; 0 0}}{\rm{.90 0}}{\rm{.05; 0}}{\rm{.03 0}}{\rm{.05 0}}{\rm{.85}}} \right];\\ > > A = Q^60\end{array}\)

\({Q^{60}} = \left[ {\begin{array}{*{20}{c}}{.7360}&{.7320}&{.7338}\\{.0876}&{.0909}&{.0894}\\{.1763}&{.1771}&{.1767}\end{array}} \right]\)

Substitute 70 for\(k\)in\({Q^k}\)to obtain\({Q^{70}}\).

\(\begin{array}{l} > > Q = \left[ {{\rm{0}}{\rm{.97 }}{\rm{.0}}{\rm{.05 0}}{\rm{.10; 0 0}}{\rm{.90 0}}{\rm{.05; 0}}{\rm{.03 0}}{\rm{.05 0}}{\rm{.85}}} \right];\\ > > A = Q^70\end{array}\)

\({Q^{70}} = \left[ {\begin{array}{*{20}{c}}{.7356}&{.7340}&{.7347}\\{.0880}&{.0893}&{.0887}\\{.1764}&{.1767}&{.1766}\end{array}} \right]\)

Substitute 80 for\(k\)in\({Q^k}\)to obtain\({Q^{80}}\).

\(\begin{array}{l} > > Q = \left[ {{\rm{0}}{\rm{.97 }}{\rm{.0}}{\rm{.05 0}}{\rm{.10; 0 0}}{\rm{.90 0}}{\rm{.05; 0}}{\rm{.03 0}}{\rm{.05 0}}{\rm{.85}}} \right];\\ > > A = Q^80\end{array}\)

\({Q^{80}} = \left[ {\begin{array}{*{20}{c}}{.7354}&{.7348}&{.7351}\\{.0881}&{.0887}&{.0884}\\{.1764}&{.1766}&{.1765}\end{array}} \right]\)

Substitute 117 for\(k\)in\({Q^k}\)to obtain\({Q^{117}}\).

\(\begin{array}{l} > > Q = \left[ {{\rm{0}}{\rm{.97 }}{\rm{.0}}{\rm{.05 0}}{\rm{.10; 0 0}}{\rm{.90 0}}{\rm{.05; 0}}{\rm{.03 0}}{\rm{.05 0}}{\rm{.85}}} \right];\\ > > A = Q^117\end{array}\)

\({Q^{117}} = \left[ {\begin{array}{*{20}{c}}{.7353}&{.7353}&{.7353}\\{.0882}&{.0882}&{.0882}\\{.1765}&{.1765}&{.1765}\end{array}} \right]\)

Thus, as\(k\)increases,\({Q^k}\)converges to the common vector\(\left[ {\begin{array}{*{20}{c}}{.7353}\\{.0882}\\{.1765}\end{array}} \right]\).

It means the steady-state vector q is\(\left[ {\begin{array}{*{20}{c}}{.7353}\\{.0882}\\{.1765}\end{array}} \right]\).

From parts (a) and (b), the\(n \times n\)stochastic matrix\(P\)converges to\(q\)(steady-state vector) or a common vector as\(k\)tends to infinity.

So,\({{\bf{x}}_k} = {p^k}{{\bf{x}}_0}\)converges to q as \(k \to \infty \).