Sums of Squares and Cross Products Matrix
This lesson introduces the sums of squares and cross products matrix (aka, SSCP matrix).
We show how to use matrix
methods to compute the SSCP matrix, using
both raw scores and
deviation scores.
Sum of Squares: Vectors
In statistics, many formulas require the calculation of
sums of squares; i.e., squaring all of the elements
in a set and then taking the sum of those squares.
Using matrix algebra, the sum of squares for all the elements of a
vector
is calculated according to the following formula:
Σ xi2 = x'x
where
x is an n x 1 column vector
of scores: x1, x2, . . . ,
xn
Σ xi2 is the sum of the squared values from
vector x
To illustrate, let's find the sum of squares for the elements of vector
x, where x' = [ 1 2 3 ].
Σ xi2 = ( 1 * 1 ) + ( 2 * 2 ) + ( 3 * 3 )
Σ xi2 = 1 + 4 + 9 = 14
Thus, the sum of the squared elements from vector x is 14.
Sums of Squares and Cross Products: Matrices
With matrices,
we can compute not only sums of squares but also sums of cross products.
For an r x c matrix,
an individual cross product is represented by
XrjXrk.
The sum of cross products between all the elements of columns
j and k is represented by
Σ XrjXrk,
summed over r.
A matrix of sums of squares and sums of cross products is represented by
X' X, as shown below.
X' X
=
|
|
Σ X12 |
Σ X1 X2 |
. . . |
Σ X1 Xc |
|
Σ X2 X1 |
Σ X22 |
. . . |
Σ X2 Xc |
. . . |
. . . |
. . . |
. . . |
Σ Xc X1 |
Σ Xc X2 |
. . . |
Σ Xc2 |
|
where
X is an r x c matrix of raw scores:
X11, X12, . . . ,
Xrc
X' X is a c x c matrix
of sums of squares and sums of cross products
Σ Xi2 is the sum of the square of all
elements in column i of matrix X
Σ Xi Xj is the sum of cross products
produced by multiplying each
element in column i of matrix X with the
corresponding element from column j and summing the result
Thus, the diagonal elements of X' X
are sums of squares, and the off-diagonal elements are cross products.
Note that the cross product matrix X' X
is a
symmetric matrix.
See problem 1 for an example
showing how to create a cross product matrix.
Sums of Squares and Deviation Scores
In the previous lesson,
we showed how to transform a matrix of raw scores into a matrix of
deviation scores. There are advantages to working with
deviation scores.
- Computations can be easier.
- Equations are often more comprehensible.
As a result, researchers often transform their
raw data into deviation scores before they calculate sums of squares and
cross products. This is such a common practice that the term
"sums of squares" has two meanings. It can refer to raw score sums of
squares or to deviation score sums of squares.
See problem 2 for an example
showing how to create a cross product matrix, using deviation scores.
Test Your Understanding
Problem 1
Using matrix A below, find the cross products matrix
defined by
matrix A' A.
A = |
|
90 |
60 |
90 |
|
90 |
90 |
30 |
60 |
60 |
60 |
60 |
60 |
90 |
30 |
30 |
30 |
|
Solution
A' A = |
|
90 |
90 |
60 |
60 |
30 |
|
60 |
90 |
60 |
60 |
30 |
90 |
30 |
60 |
90 |
30 |
|
|
|
90 |
60 |
90 |
|
90 |
90 |
30 |
60 |
60 |
60 |
60 |
60 |
90 |
30 |
30 |
30 |
|
A' A = |
|
24,300 |
21,600 |
20,700 |
|
21,600 |
19,800 |
18,000 |
20,700 |
18,000 |
21,600 |
|
Problem 2
Matrix A, shown below, is a matrix of raw scores.
Find the deviation score sums of squares matrix produced from matrix
A; that is, find
matrix a' a.
A = |
|
90 |
60 |
90 |
|
90 |
90 |
30 |
60 |
60 |
60 |
60 |
60 |
90 |
30 |
30 |
30 |
|
Solution
First, we transform raw score matrix A into
deviation score matrix a, as shown below.
Previously, we described
how to transform raw scores to deviation scores.
We repeat the transformation formula below, and then we make
the transformation.
a =
A -
11'A
( 1 / r )
where
1 is an 5 x 1 column
vector
of ones
a is an 5 x 3 matrix
of deviation scores: a11,
a12, . . . ,
a53
A is an 5 x 3 matrix
of raw scores: A11,
A12, . . . ,
A53
r is the number of rows in matrix A
a = |
|
90 |
60 |
90 |
|
90 |
90 |
30 |
60 |
60 |
60 |
60 |
60 |
90 |
30 |
30 |
30 |
|
- |
|
1 |
1 |
1 |
1 |
1 |
|
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
1 |
|
|
|
90 |
60 |
90 |
|
90 |
90 |
30 |
60 |
60 |
60 |
60 |
60 |
90 |
30 |
30 |
30 |
|
|
( 1/5 ) |
a = |
|
90 |
60 |
90 |
|
90 |
90 |
30 |
60 |
60 |
60 |
60 |
60 |
90 |
30 |
30 |
30 |
|
- |
|
66 |
60 |
60 |
|
66 |
60 |
60 |
66 |
60 |
60 |
66 |
60 |
60 |
66 |
60 |
60 |
|
a = |
|
24 |
0 |
30 |
|
24 |
30 |
-30 |
-6 |
0 |
0 |
-6 |
0 |
30 |
-36 |
-30 |
-30 |
|
Then, to find the deviation score sums of square matrix, we simply
compute a'a, as shown below.
a' a = |
|
24 |
24 |
-6 |
-6 |
36 |
|
0 |
30 |
0 |
0 |
-30 |
30 |
-30 |
0 |
30 |
-30 |
|
|
|
24 |
0 |
30 |
|
24 |
30 |
-30 |
-6 |
0 |
0 |
-6 |
0 |
30 |
-36 |
-30 |
-30 |
|
a' a = |
|
2520 |
1800 |
900 |
|
1800 |
1800 |
0 |
900 |
0 |
3600 |
|