Variance-Covariance Matrix

This lesson explains how to use matrix methods to generate a variance-covariance matrix from a matrix of raw data.

Variance

Variance is a measure of the variability or spread in a set of data. Mathematically, it is the average squared deviation from the mean score. We use the following formula to compute variance.

Var(X) = Σ ( X_i - X )² / N = Σ x_i² / N

where

N is the number of scores in a set of scores
X is the mean of the N scores.
X_i is the ith raw score in the set of scores
x_i is the ith deviation score in the set of scores
Var(X) is the variance of all the scores in the set

Covariance

Covariance is a measure of the extent to which corresponding elements from two sets of ordered data move in the same direction. We use the following formula to compute covariance.

Cov(X, Y) = Σ ( X_i - X ) ( Y_i - Y ) / N = Σ x_iy_i / N

where

N is the number of scores in each set of data
X is the mean of the N scores in the first data set
X_i is the ithe raw score in the first set of scores
x_i is the ith deviation score in the first set of scores
Y is the mean of the N scores in the second data set
Y_i is the ithe raw score in the second set of scores
y_i is the ith deviation score in the second set of scores
Cov(X, Y) is the covariance of corresponding scores in the two sets of data

Variance-Covariance Matrix

Variance and covariance are often displayed together in a variance-covariance matrix, (aka, a covariance matrix). The variances appear along the diagonal and covariances appear in the off-diagonal elements, as shown below.

where

V is a c x c variance-covariance matrix
N is the number of scores in each of the c data sets
x_i is a deviation score from the ith data set
Σ x_i² / N is the variance of elements from the ith data set
Σ x_i x_j / N is the covariance for elements from the ith and jth data sets

How to Create a Variance-Covariance Matrix

Suppose X is an n x k matrix holding ordered sets of raw data. For example, matrix X might display the scores on k tests for n students, as shown in Problem 1.

Starting with the raw data of matrix X, you can create a variance-covariance matrix to show the variance within each column and the covariance between columns. Here's how.

• Transform the raw scores from matrix X into deviation scores for matrix x.

  x = X - 11'X ( 1 / n )

  where

  1 is an n x 1 column vector of ones
  x is an n x k matrix of deviation scores: x₁₁, x₁₂, . . . , x_nk
  X is an n x k matrix of raw scores: X₁₁, X₁₂, . . . , X_nk

• Compute x'x, the k x k deviation sums of squares and cross products matrix for x.

• Then, divide each term in the deviation sums of squares and cross product matrix by n to create the variance-covariance matrix. That is,

  V = x'x ( 1 / n )

  where

  V is a k x k variance-covariance matrix
  x'x is the deviation sums of squares and cross product matrix
  n is the number of scores in each column of the original matrix X

In the next section, read Problem 1 for an example showing how to turn raw data into a variance-covariance matrix.