Hypothesis Test: Difference Between Paired Means

This lesson explains how to conduct a hypothesis test for the difference between paired means. The test procedure, called the matched-pairs t-test, is appropriate when the following conditions are met:

  • The sampling method for each sample is simple random sampling.
  • The test is conducted on paired data. (As a result, the data sets are not independent.)
  • Each sample is drawn from a normal or near-normal population. Generally, the sampling distribution will be approximately normal if any of the following conditions apply.

    • The population distribution is normal.
    • The sample data are symmetric, unimodal, without outliers, and the sample size is 15 or less.
    • The sample data are slightly skewed, unimodal, without outliers, and the sample size is 16 to 40.
    • The sample size is greater than 40, without outliers.

This approach consists of four steps: (1) state the hypotheses, (2) formulate an analysis plan, (3) analyze sample data, and (4) interpret results.

State the Hypotheses

Every hypothesis test requires the analyst to state a null hypothesis and an alternative hypothesis. The hypotheses are stated in such a way that they are mutually exclusive. That is, if one is true, the other must be false; and vice versa.

The hypotheses concern a new variable d, which is based on the difference between paired values from two data sets.

d = x1 - x2

where x1 is the value of variable x in the first data set, and x2 is the value of the variable from the second data set that is paired with x1.

The table below shows three sets of null and alternative hypotheses. Each makes a statement about how the true difference in population values μd is related to some hypothesized value D. (In the table, the symbol ≠ means " not equal to ".)

Set Null hypothesis Alternative hypothesis Number of tails
1 μd= D μd ≠ D 2
2 μd > D μd < D 1
3 μd < D μd > D 1

The first set of hypotheses (Set 1) is an example of a two-tailed test, since an extreme value on either side of the sampling distribution would cause a researcher to reject the null hypothesis. The other two sets of hypotheses (Sets 2 and 3) are one-tailed tests, since an extreme value on only one side of the sampling distribution would cause a researcher to reject the null hypothesis.

Formulate an Analysis Plan

The analysis plan describes how to use sample data to accept or reject the null hypothesis. It should specify the following elements.

  • Significance level. Often, researchers choose significance levels equal to 0.01, 0.05, or 0.10; but any value between 0 and 1 can be used.

  • Test method. Use the matched-pairs t-test to determine whether the difference between sample means for paired data is significantly different from the hypothesized difference between population means.

Analyze Sample Data

Using sample data, find the standard deviation, standard error, degrees of freedom, test statistic, and the P-value associated with the test statistic.

  • Standard deviation. Compute the standard deviation (sd) of the differences computed from n matched pairs.

    sd = sqrt [ (Σ(di - d)2 / (n - 1) ]

    where di is the difference for pair i, d is the sample mean of the differences, and n is the number of paired values.

  • Standard error. Compute the standard error (SE) of the sampling distribution of d.

    SE = sd * sqrt{ ( 1/n ) * ( 1 - n/N ) * [ N / ( N - 1 ) ] }

    where sd is the standard deviation of the sample difference, N is the population size, and n is the sample size. When the population size is much larger (at least 10 times larger) than the sample size, the standard error can be approximated by:

    SE = sd / sqrt( n )

  • Degrees of freedom. The degrees of freedom (DF) is: DF = n - 1 .

  • Test statistic. The test statistic is a t-score (t) defined by the following equation.

    t = [ (x1 - x2) - D ] / SE = (d - D) / SE

    where x1 is the mean of sample 1, x2 is the mean of sample 2, d is the mean difference between paired values in the sample, D is the hypothesized difference between population means, and SE is the standard error.

  • P-value. The P-value is the probability of observing a sample statistic as extreme as the test statistic. Since the test statistic is a t-score, use the t Distribution Calculator to assess the probability associated with the t-score, having the degrees of freedom computed above. (See the sample problem at the end of this lesson for guidance on how this is done.)

Interpret Results

If the sample findings are unlikely, given the null hypothesis, the researcher rejects the null hypothesis. Typically, this involves comparing the P-value to the significance level, and rejecting the null hypothesis when the P-value is less than the significance level.

Test Your Understanding of This Lesson

Problem

Forty-four sixth graders were randomly selected from a school district. Then, they were divided into 22 matched pairs, each pair having equal IQ's. One member of each pair was randomly selected to receive special training. Then, all of the students were given an IQ test. Test results are summarized below.

Pair Training No training Difference, d (d - d)2
1 95 90 5 16
2 89 85 4 9
3 76 73 3 4
4 92 90 2 1
5 91 90 1 0
6 53 53 0 1
7 67 68 -1 4
8 88 90 -2 9
9 75 78 -3 16
10 85 89 -4 25
11 90 95 -5 36
Pair Training No training Difference, d (d - d)2
12 85 83 2 1
13 87 83 4 9
14 85 83 2 1
15 85 82 3 4
16 68 65 3 4
17 81 79 2 1
18 84 83 1 0
19 71 60 11 100
20 46 47 -1 4
21 75 77 -2 9
22 80 83 -3 16

Σ(d - d)2 = 270
d = 1

Do these results provide evidence that the special training helped or hurt student performance? Use an 0.05 level of significance. Assume that the mean differences are approximately normally distributed.

Solution

The solution to this problem takes four steps: (1) state the hypotheses, (2) formulate an analysis plan, (3) analyze sample data, and (4) interpret results. We work through those steps below:

  • State the hypotheses. The first step is to state the null hypothesis and an alternative hypothesis.

    Null hypothesis: μd = 0
    Alternative hypothesis: μd ≠ 0

    Note that these hypotheses constitute a two-tailed test. The null hypothesis will be rejected if the difference between sample means is too big or if it is too small.

  • Formulate an analysis plan. For this analysis, the significance level is 0.05. Using sample data, we will conduct a matched-pairs t-test of the null hypothesis.

  • Analyze sample data. Using sample data, we compute the standard deviation of the differences (s), the standard error (SE) of the mean difference, the degrees of freedom (DF), and the t-score test statistic (t).

    s = sqrt [ (Σ(di - d)2 / (n - 1) ] = sqrt[ 270/(22-1) ] = sqrt(12.857) = 3.586

    SE = s / sqrt(n) = 3.586 / [ sqrt(22) ] = 3.586/4.69 = 0.765

    DF = n - 1 = 22 -1 = 21

    t = [ (x1 - x2) - D ] / SE = (d - D)/ SE = (1 - 0)/0.765 = 1.307

    where di is the observed difference for pair i, d is mean difference between sample pairs, D is the hypothesized mean difference between population pairs, and n is the number of pairs.

    Since we have a two-tailed test, the P-value is the probability that a t-score having 21 degrees of freedom is more extreme than 1.307; that is, less than -1.307 or greater than 1.307.

    We use the t Distribution Calculator to find P(t < -1.307) = 0.103, and P(t > 1.307) = 0.103. Thus, the P-value = 0.103 + 0.103 = 0.206.

  • Interpret results. Since the P-value (0.206) is greater than the significance level (0.05), we cannot reject the null hypothesis.

Note: If you use this approach on an exam, you may also want to mention why this approach is appropriate. Specifically, the approach is appropriate because the sampling method was simple random sampling, the samples consisted of paired data, and the mean differences were normally distributed. In addition, we used the approximation formula to compute the standard error, since the sample size was small relative to the population size.