Orthogonal Comparisons and ANOVA

In the previous lesson, we learned that researchers use comparisons to address follow-up questions that are not answered by a standard, omnibus analysis of variance.

This lesson focuses on a special type of comparison, called an orthogonal comparison.

What is a Comparison?

To be sure everyone is on the same page, let's review what we know about comparisons. A comparison is a weighted sum of mean scores. Mathematically, a comparison can be expressed as:

In addition, all comparisons are subject to the following constraint:

In the equations above, L is the value of the comparison, c_j is a coefficient (weight) for treatment j, X_j is the mean score for treatment j, n_j is the number of subjects assigned to treatment j , and k is the number of treatment groups.

With balanced designs (i.e., designs in which sample size is constant across treatment groups), the necessary condition for a comparison reduces to:

Σ c_j = 0

What is an Orthogonal Comparison?

Two comparisons (L₁ and L₂) are orthogonal if the following is true:

Σ n_j c_1j c_2j  = 0

where n_j is the sample size in treatment group j, c_1j is a coefficient for treatment j in comparison L₁, and c_2j is a coefficient for treatment j in comparison L₂.

With balanced designs (i.e., designs in which sample size is constant across treatment groups), the necessary condition for two comparisons to be orthogonal reduces to:

Σ c_1j c_2j  = 0

For any experiment with three or more treatment groups, it is possible to define different sets of orthogonal comparisons. However, each set of orthogonal comparisons can have a maximum of k - 1 comparisons, where k is the number of treatment groups.

Orthogonal Comparisons and Sums of Squares

The sum of squares for an orthogonal comparison is computed in the same way as the sum of squares for an ordinary comparison. When Σ n_jc_j = 0, the sum of squares for a comparison can be computed from the following formula:

SS_i = ( Σ n_jc_ij X_j )² / Σ n_jc²_ij

where SS_i is the sum of squares for comparison L_i , n_j is the sample size in Group j , c_ij is the coefficient (weight) for level j in the formula for comparison L_i, and X_j is the mean score for Group j .

With a balanced design, the sum of squares for a comparison ( L_i ) can be computed from a simpler formula:

SS_i = n * L_i² / Σ c²_ij

where SS_i is the sum of squares for comparison L_i , L_i is the value of the comparison, n is the sample size in each group, and c_ij is the coefficient (weight) for level j in the formula for comparison L_i.

The sums of squares for a complete set of orthogonal comparisons (SS_j) are related to the sum of squares for the treatment effect (SS_T) in a standard analysis of variance, as shown below:

SS_T = SS₁ + SS₂ + ... + SS_k-1

A treatment effect with k levels has a maximum of k - 1 orthogonal comparisons. There is an additive relationship between the sum of square for a treatment effect and the sums of squares for k - 1 orthogonal comparisons. The sum of squares for the treatment effect is equal to the sum of sums of squares for k - 1 orthogonal comparisons.

Orthogonal Comparisons and Uncorrelated Data

When conducting multiple follow-up tests to supplement an omnibus analysis of variance, it is sometimes desirable to formulate hypotheses that can be tested using uncorrelated data from the experiment.

When the data used to test one hypothesis are uncorrelated with the data used to test another hypothesis, the hypothesis tests are independent. It is as if each hypothesis test were conducted using data from different experiments.

So, how do you distinguish hypotheses that can be tested using uncorrelated data from those that cannot? You look for orthogonal comparisons. Hypotheses represented by orthogonal comparisons use uncorrelated data (aka, nonoverlapping data) for significance tests. Hypotheses represented by nonorthogonal comparisons use correlated data (aka, overlapping data).

Test Your Understanding

Problem 1

You are conducting a single-factor experiment with four treatment groups. Here are five comparisons relevant to that experiment:

• L₁ = X₁ - X₂
• L₂ = X₂ - X₃
• L₃ = X₃ - X₄
• L₄ = 0.5X₁ + 0.5X₂ - 0.5X₃ - 0.5X₄
• L₅ = 0.5X₁ - 0.5X₂ + 0.5X₃ - 0.5X₄

In which of the following sets of comparisons are all three comparisons orthogonal? (Assume a balanced design; i.e., equal sample size in each group.)

(A) L₁, L₂, and L₃
(B) L₁, L₂, and L₄
(C) L₁, L₂, and L₅
(D) L₁, L₃, and L₄
(E) L₁, L₃, and L₅

Solution

The correct answer is (D). With a balanced design, the necessary condition for two comparisons to be orthogonal is:

Σ c_1j c_2j  = 0

The table below shows weights (c_j) for each comparison:

Comparison	c1	c2	c3	c4
L1	1	-1	0	0
L2	0	1	-1	0
L3	0	0	1	-1
L4	0.5	0.5	-0.5	-0.5
L5	0.5	-0.5	0.5	-0.5

From the table, it is evident that the necessary condition for orthogonality is satisfied for the set of comparisons composed of L₁, L₃, and L₄. That is,

Σ c_1j c_3j  = (1)(0) + (-1)(0) + (0)(1) + (0)(-1) = 0

Σ c_1j c_4j  = (1)(0.5) + (-1)(0.5) + (0)(-0.5) + (0)(-0.5) = 0

Σ c_3j c_4j  = (0)(0.5) + (0)(0.5) + (1)(-0.5) + (1)(-0.5) = 0

The necessary condition for orthogonality is not satisfied for any of the other sets of comparisons. For example, comparisons L₁ and L₂ are not orthogonal because:

Σ c_1j c_2j  = (1)(0) + (-1)(1) + (0)(-1) + (0)(0) = -1

Because Σ c_1j c_2j is not equal to zero, we know that L₁ and L₂ are not orthogonal; so options A, B, and C cannot be correct answers to this problem.

Similarly, comparisons L₁ and L₅ are not orthogonal because:

Σ c_1j c_5j  = (1)(0.5) + (-1)(-0.5) + (0)(0.5) + (0)(-0.5) = 1

Because Σ c_1j c_5j is not equal to zero, option E cannot be a correct answer to this problem.

Problem 2

You are conducting a single-factor experiment with three treatment groups, equal sample sizes in each group. And you are interested in three comparisons:

• L₁ = X₁ - X₂
• L₂ = X₂ - X₃
• L₃ = X₁ - 0.5X₂ - 0.5X₃

The sum of squares for each comparison is shown below:

SS1	SS2	SS3
10	20	30

What is the sum of squares for the treatment effect in this experiment?

(A) 20
(B) 30
(C) 40
(D) 50
(E) 60

Solution

The correct answer is (D). Here's the logic for solving this problem:

• There is an additive relationship between the sum of squares for a treatment effect and the sums of squares for k - 1 orthogonal comparisons. That is,

  SS_T = SS₁ + SS₂ + ... + SS_k-1

  where SS_T is the sum of squares for the treatment effect, SS_i is the sum of squares for comparison i, and k is the number of treatment groups in the experiment.
• This experiment has three treatment groups; thus, k = 3.
• If we can identify k - 1 orthogonal comparisons, we can use the formula above to compute the sum of squares for the treatment effect.

With a balanced design, the necessary condition for two comparisons to be orthogonal is:

Σ c_1j c_2j  = 0

The table below shows weights (c_j) for all three comparisons:

Comparison	c1	c2	c3
L1	1	-1	0
L2	0	1	-1
L3	1	-0.5	-0.5

Comparisons L₁ and L₂ are not orthogonal because Σ c_1j c_2j ≠ 0. Similarly, comparisons L₁ and L₃ are not orthogonal because Σ c_1j c_3j ≠ 0.

Σ c_1j c_2j  = (1)(0) + (-1)(1) + (0)(-1) = -1

Σ c_1j c_3j  = (1)(1) + (-1)(-0.5) + (0)(-0.5) = 1.5

However, comparisons L₂ and L₂ are orthogonal because Σ c_2j c_3j = 0, as shown below:

Σ c_2j c_3j  = (0)(1) + (1)(-0.5) + (-1)(-0.5) = 0

Because comparisons L₂ and L₃ represent k - 1 orthogonal comparisons. we can apply the formula below to compute the sum of squares for the treatment effect:

SS_T = SS₂ + SS₃

SS_T = 20 + 30 = 50