Exercises 19 and 20 involve a design matrix \(X\) with two or more columns and a least-squares solution \(\hat \beta \) of \({\bf{y}} = X\beta \). Consider the following numbers.

(i) \({\left\| {X\hat \beta } \right\|^2}\)—the sum of the squares of the “regression term.” Denote this number by .

(ii) \({\left\| {{\bf{y}} - X\hat \beta } \right\|^2}\)—the sum of the squares for error term. Denote this number by \(SS\left( E \right)\).

(iii) \({\left\| {\bf{y}} \right\|^2}\)—the “total” sum of the squares of the \(y\)-values. Denote this number by \(SS\left( T \right)\).

Every statistics text that discusses regression and the linear model \(y = X\beta + \in \) introduces these numbers, though terminology and notation vary somewhat. To simplify matters, assume that the mean of the -values is zero. In this case, \(SS\left( T \right)\) is proportional to what is called the variance of the set of -values.

19. Justify the equation \(SS\left( T \right) = SS\left( R \right) + SS\left( E \right)\). (Hint: Use a theorem, and explain why the hypotheses of the theorem are satisfied.) This equation is extremely important in statistics, both in regression theory and in the analysis of variance.

Step by step solution

01

Find \(SS\left( T \right)\)

The given residual vector is \( \in = {\bf{y}} - X\hat \beta \) which is orthogonal to \(\text{Col}X\), while \({\bf{\hat y}} = X\hat \beta \) is in \({\rm{Col}}X\).

As, \( \in = {\bf{y}} - X\hat \beta \) and \({\bf{\hat y}} = X\hat \beta \) are orthogonal, apply the orthogonal theorem and find \(\).

\(\begin{aligned}SS\left( T \right) &= {\left\| {\bf{y}} \right\|^2}\\ &= {\left\| {{\bf{\hat y}} + \in } \right\|^2}\\ &= {\left\| {{\bf{\hat y}}} \right\|^2} + {\left\| \in \right\|^2}\end{aligned}\)

Use \({\bf{\hat y}} = X\hat \beta \) and \( \in = {\bf{y}} - X\hat \beta \) into the obtained expression.

\(SS\left( T \right) = {\left\| {X\hat \beta } \right\|^2} + {\left\| {{\bf{y}} - X\hat \beta } \right\|^2}{\rm{ }}\left( 1 \right)\)

02

Find \(SS\left( R \right) + SS\left( E \right)\)

Find\(SS\left( R \right) + SS\left( E \right)\).

\(SS\left( R \right) + SS\left( E \right) = {\left\| {X\hat \beta } \right\|^2} + {\left\| {{\bf{y}} - X\hat \beta } \right\|^2}{\rm{ }}\left( 2 \right)\)

From equations (1) and (2),

\(SS\left( T \right) = SS\left( R \right) + SS\left( E \right)\)

Hence, the equation \(SS\left( T \right) = SS\left( R \right) + SS\left( E \right)\) is justified.

Unlock Step-by-Step Solutions & Ace Your Exams!

• Full Textbook Solutions

  Get detailed explanations and key concepts

• Unlimited Al creation

  Al flashcards, explanations, exams and more...

• Ads-free access

  To over 500 millions flashcards

• Money-back guarantee

  We refund you if you fail your exam.

Start your free trial

Over 30 million students worldwide already upgrade their learning with Vaia!