Power of a Hypothesis Test
The probability of not committing a
Type II error is called the power of
a hypothesis test.
Effect Size
To compute the power of the test, one offers an alternative
view about the "true" value of the population parameter,
assuming that the null hypothesis is false. The
effect size is the difference
between the true value and the value specified in the null
hypothesis.
Effect size = True value - Hypothesized value
For example, suppose the null hypothesis states that a population
mean is equal to 100. A researcher might ask: What is the probability
of rejecting the null hypothesis if the true population mean is
equal to 90? In this example, the
effect size would be 90 - 100, which equals -10.
Factors That Affect Power
The power of a hypothesis test is affected by three factors.
- Sample size (n). Other things being equal, the greater
the sample size, the greater the power of the test.
- Significance level (α). The lower the significance level,
the lower the power of the test. If you reduce the significance level (e.g., from 0.05 to 0.01),
the
region of acceptance gets bigger. As a result, you are less likely to
reject the null hypothesis. This means you are less likely to
reject the null hypothesis when it is false, so you are more likely to
make a Type II error. In short, the power of the test is reduced when you reduce the significance level; and vice versa.
- The "true" value of the parameter being tested. The greater the
difference between the "true" value of a parameter and the value
specified in the null hypothesis, the greater the power of
the test. That is, the greater the effect size,
the greater the power of the test.
Test Your Understanding
Problem 1
Other things being equal, which of the following actions will
reduce the power of a hypothesis test?
I. Increasing sample size.
II. Changing the significance level from 0.01 to 0.05.
III. Increasing beta, the probability of a Type II error.
(A) I only
(B) II only
(C) III only
(D) All of the above
(E) None of the above
Solution
The correct answer is (C). Increasing sample size makes the hypothesis
test more sensitive - more likely to reject the null hypothesis
when it is, in fact, false. Changing the significance level from 0.01 to 0.05
makes the
region of acceptance smaller, which makes the hypothesis test more
likely to reject the null hypothesis, thus increasing the
power of the test. Since, by definition, power is equal to one
minus beta, the power of a test will get smaller as beta gets
bigger.
Problem 2
Suppose a researcher conducts an experiment to test a hypothesis.
If she doubles her sample size, which of the following will
increase?
I. The power of the hypothesis test.
II. The effect size of the hypothesis test.
III. The probability of making a Type II error.
(A) I only
(B) II only
(C) III only
(D) All of the above
(E) None of the above
Solution
The correct answer is (A). Increasing sample size makes the hypothesis
test more sensitive - more likely to reject the null hypothesis
when it is, in fact, false. Thus, it increases the power of the
test. The effect size is not affected by sample size. And the
probability of making a
Type II error gets smaller, not bigger, as sample size
increases.