VarianceCovariance Matrix
This lesson explains how to use matrix methods to generate a
variancecovariance matrix from a matrix of raw data.
Variance
Variance is a measure of the variability or spread in
a set of data.
Mathematically, it is the average squared
deviation from the mean score. We use the following formula
to compute population variance.
Var(X) =
Σ ( X_{i} 
X )^{2} / N =
Σ x_{i}^{2} / N
where
N is the number of scores in a set of scores
X is the
mean
of the N scores.
X_{i} is the ith raw score in the set of scores
x_{i} is the ith deviation score in the set of
scores
Var(X) is the variance of all the scores in the set
Covariance
Covariance is a measure of the extent to which
corresponding elements from two sets of ordered data move in the same direction.
We use the following formula to compute population covariance.
Cov(X, Y) =
Σ ( X_{i}  X )
( Y_{i}  Y ) / N
= Σ x_{i}y_{i} / N
where
N is the number of scores in each set of data
X is the
mean
of the N scores in the first data set
X_{i} is the ithe raw score in the first set of
scores
x_{i} is the ith deviation score in the first set of
scores
Y is the
mean
of the N scores in the second data set
Y_{i} is the ithe raw score in the second set
of scores
y_{i} is the ith deviation score in the second set of
scores
Cov(X, Y) is the covariance of corresponding scores in the
two sets of data
VarianceCovariance Matrix
Variance and covariance are often displayed together in a
variancecovariance
matrix,
(aka, a covariance matrix).
The variances appear along the diagonal and
covariances appear in the offdiagonal elements, as shown below.
V =


Σ x_{1}^{2} / N 
Σ x_{1} x_{2} / N 
. . . 
Σ x_{1} x_{c} / N 

Σ x_{2} x_{1} / N 
Σ x_{2}^{2} / N 
. . . 
Σ x_{2} x_{c} / N 
. . . 
. . . 
. . . 
. . . 
Σ x_{c} x_{1} / N 
Σ x_{c} x_{2} / N 
. . . 
Σ x_{c}^{2} / N 

where
V is a c x c variancecovariance matrix
N is the number of scores in each of the c data sets
x_{i} is a
deviation score
from the ith data set
Σ x_{i}^{2} / N is the variance of elements from the
ith data set
Σ x_{i} x_{j} / N is the covariance for
elements from the ith and jth data sets
How to Create a VarianceCovariance Matrix
Suppose X is an n x k matrix
holding ordered sets of raw data.
For example, matrix X might display the scores
on k tests for n students, as shown in
Problem 1.
Starting with the raw data of matrix X,
you can create a variancecovariance matrix to
show the variance within each column and the covariance between columns.
Here's how.
Transform the raw scores from matrix X
into deviation scores for matrix x.
x =
X 
11'X
( 1 / n )
where
1 is an n x 1 column
vector
of ones
x is an n x k matrix
of deviation scores: x_{1}_{1},
x_{1}_{2}, . . . ,
x_{n}_{k}
X is an n x k matrix
of raw scores: X_{1}_{1},
X_{1}_{2}, . . . ,
X_{n}_{k}
Compute x'x, the k x k
deviation sums of squares and cross products matrix for
x.
Then, divide each term in the deviation sums of squares and cross product
matrix by n to create the variancecovariance matrix.
That is,
V =
x'x ( 1 / n )
where
V is a k x k variancecovariance matrix
x'x
is the deviation sums of squares and cross product matrix
n is the number of scores in each column of the original matrix
X
In the next section, read Problem 1
for an example showing how to turn raw data into a
variancecovariance matrix.
Test Your Understanding
Problem 1
The table below displays scores on math, English, and art tests
for 5 students.
Student 
Math 
English 
Art 
1 
90 
60 
90 
2 
90 
90 
30 
3 
60 
60 
60 
4 
60 
60 
90 
5 
30 
30 
30 
Note that data from the table can be represented in
matrix A, where each column in the matrix shows scores
on a test and each row shows scores for a student.
A =


90 
60 
90 

90 
90 
30 
60 
60 
60 
60 
60 
90 
30 
30 
30 

Given the data represented in matrix A,
compute the variance of each test and the covariance between the tests.
Solution
The solution involves a threestep process.
 First, we transform the raw scores in matrix A
to deviation scores in matrix a,
using the transformation formula described at
how to transform raw scores to deviation scores.
a =
A 
11'A
( 1 / n )
where
1 is an 5 x 1 column
vector
of ones
a is an 5 x 3 matrix
of deviation scores: a_{1}_{1},
a_{1}_{2}, . . . ,
a_{5}_{3}
A is an 5 x 3 matrix
of raw scores: A_{1}_{1},
A_{1}_{2}, . . . ,
A_{5}_{3}
n is the number of rows in matrix A

90 
60 
90 

90 
90 
30 
60 
60 
60 
60 
60 
90 
30 
30 
30 

1 
1 
1 
1 
1 

1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 
1 

90 
60 
90 

90 
90 
30 
60 
60 
60 
60 
60 
90 
30 
30 
30 
a = 

90 
60 
90 

90 
90 
30 
60 
60 
60 
60 
60 
90 
30 
30 
30 

 

66 
60 
60 

66 
60 
60 
66 
60 
60 
66 
60 
60 
66 
60 
60 

a = 

24 
0 
30 

24 
30 
30 
6 
0 
0 
6 
0 
30 
36 
30 
30 

a' a = 

24 
24 
6 
6 
36 

0 
30 
0 
0 
30 
30 
30 
0 
30 
30 



24 
0 
30 

24 
30 
30 
6 
0 
0 
6 
0 
30 
36 
30 
30 

a' a = 

2520 
1800 
900 

1800 
1800 
0 
900 
0 
3600 

And finally, to create the variancecovariance matrix, we
divide each element in the deviation sum of squares matrix by n,
as shown below.
V
=
a' a / n
V
=


2520/5 
1800/5 
900/5 

1800/5 
1800/5 
0/5 
900/5 
0/5 
3600/5 

V
=


504 
360 
180 

360 
360 
0 
180 
0 
720 

We can interpret the variance and covariance statistics in
matrix V to understand how the various test
scores vary and covary.
 Shown in red along the diagonal, we see the variance of scores for each
test. The art test has the biggest variance (720);
and the English test, the smallest (360). So we can say that
art test scores are more variable than English test scores.
 The covariance is displayed in black in the offdiagonal elements
of matrix V.
 The covariance
between math and English is positive (360), and the covariance between
math and art is positive (180). This means the scores tend to covary
in a positive way. As scores on math go up, scores on art and
English also tend to go up; and vice versa.
 The covariance between English and art, however, is zero. This
means there tends to be no predictable relationship between the
movement of English and art scores.
If the covariance between any tests had been negative, it would have meant
that the test scores on those tests tend to move in opposite directions.
That is, students with relatively high scores on the first test would tend to
have relatively low scores on the second test.