How to Find the Power of a Statistical Test

When a researcher designs a study to test a hypothesis, he/she should compute the power of the test (i.e., the likelihood of avoiding a Type II error).

How to Compute the Power of a Hypothesis Test

To compute the power of a hypothesis test, use the following three-step procedure.

• Define the region of acceptance. Previously, we showed how to compute the region of acceptance for a hypothesis test.
• Specify the critical parameter value. The critical parameter value is an alternative to the value specified in the null hypothesis. The difference between the critical parameter value and the value from the null hypothesis is called the effect size. That is, the effect size is equal to the critical parameter value minus the value from the null hypothesis.
• Compute power. Assume that the true population parameter is equal to the critical parameter value, rather than the value specified in the null hypothesis. Based on that assumption, compute the probability that the sample estimate of the population parameter will fall outside the region of acceptance. That probability is the power of the test.

The following examples illustrate how this works. The first example involves a mean score; and the second example, a proportion.

Example 1: Power of the Hypothesis Test of a Mean Score

Two inventors have developed a new, energy-efficient lawn mower engine. One inventor says that the engine will run continuously for 5 hours (300 minutes) on a single ounce of regular gasoline. Suppose a random sample of 50 engines is tested. The engines run for an average of 295 minutes, with a standard deviation of 20 minutes. The inventor tests the null hypothesis that the mean run time is 300 minutes against the alternative hypothesis that the mean run time is not 300 minutes, using a 0.05 level of significance.

The other inventor says that the new engine will run continuously for only 290 minutes on a ounce of gasoline. Find the power of the test to reject the null hypothesis, if the second inventor is correct.

Solution: The steps required to compute power are presented below.

• Define the region of acceptance. In a previous lesson, we showed that the region of acceptance for this problem consists of the values between 294.46 and 305.54 (see previous lesson).
• Specify the critical parameter value. The null hypothesis tests the hypothesis that the run time of the engine is 300 minutes. We are interested in determining the probability that the hypothesis test will reject the null hypothesis, if the true run time is actually 290 minutes. Therefore, the critical parameter value is 290. (Another way to express the critical parameter value is through effect size. The effect size is equal to the critical parameter value minus the hypothesized value. Thus, effect size is equal to 290 - 300 or -10.)
• Compute power. The power of the test is the probability of rejecting the null hypothesis, assuming that the true population mean is equal to the critical parameter value. Since the region of acceptance is 294.46 to 305.54, the null hypothesis will be rejected when the sampled run time is less than 294.46 or greater than 305.54.

  Therefore, we need to compute the probability that the sampled run time will be less than 294.46 or greater than 305.54. To do this, we make the following assumptions:

  • The sampling distribution of the mean is normally distributed. (Because the sample size is relatively large, this assumption can be justified by the central limit theorem.)
  • The mean of the sampling distribution is the critical parameter value, 290.
  • The standard error of the sampling distribution is 2.83. The standard error of the sampling distribution was computed in a previous lesson (see previous lesson).

  Given these assumptions, we first assess the probability that the sample run time will be less than 294.46. This is easy to do, using the Normal Calculator. We enter the following values into the calculator: normal random variable = 294.46; mean = 290; and standard deviation = 2.83. Given these inputs, we find that the cumulative probability is 0.942. This means the probability that the sample mean will be less than 294.46 is 0.942.

  Next, we assess the probability that the sample mean is greater than 305.54. Again, we use the Normal Calculator. We enter the following values into the calculator: normal random variable = 305.54; mean = 290; and standard deviation = 2.83. Given these inputs, we find that the probability that the sample mean is less than 305.54 (i.e., the cumulative probability) is 1.0. Thus, the probability that the sample mean is greater than 305.54 is 1 - 1.0 or 0.0.

  The power of the test is the sum of these probabilities: 0.942 + 0.0 = 0.942. This means that if the true average run time of the new engine were 290 minutes, we would correctly reject the hypothesis that the run time was 300 minutes 94.2 percent of the time. Hence, the probability of a Type II error would be very small. Specifically, it would be 1 minus 0.942 or 0.058.

Example 2: Power of the Hypothesis Test of a Proportion

A major corporation offers a large bonus to all of its employees if at least 80 percent of the corporation's 1,000,000 customers are very satisfied. The company conducts a survey of 100 randomly sampled customers to determine whether or not to pay the bonus. The null hypothesis states that the proportion of very satisfied customers is at least 0.80. If the null hypothesis cannot be rejected, given a significance level of 0.05, the company pays the bonus.

Suppose the true proportion of satisfied customers is 0.75. Find the power of the test to reject the null hypothesis.

Solution: The steps required to compute power are presented below.

• Define the region of acceptance. In a previous lesson, we showed that the region of acceptance for this problem consists of the values between 0.734 and 1.00. (see previous lesson).
• Specify the critical parameter value. The null hypothesis tests the hypothesis that the proportion of very satisfied customers is 0.80. We are interested in determining the probability that the hypothesis test will reject the null hypothesis, if the true satisfaction level is 0.75. Therefore, the critical parameter value is 0.75. (Another way to express the critical parameter value is through effect size. The effect size is equal to the critical parameter value minus the hypothesized value. Thus, effect size is equal to [0.75 - 0.80] or - 0.05.)
• Compute power. The power of the test is the probability of rejecting the null hypothesis, assuming that the true population proportion is equal to the critical parameter value. Since the region of acceptance is 0.734 to 1.00, the null hypothesis will be rejected when the sample proportion is less than 0.734.

  Therefore, we need to compute the probability that the sample proportion will be less than 0.734. To do this, we take the following steps:

  • Assume that the sampling distribution of the mean is normally distributed. (Because the sample size is relatively large, this assumption can be justified by the central limit theorem.)
  • Assume that the mean of the sampling distribution is the critical parameter value, 0.75. (This assumption is justified because, for the purpose of calculating power, we assume that the true population proportion is equal to the critical parameter value. And the mean of all possible sample proportions is equal to the population proportion. Hence, the mean of the sampling distribution is equal to the critical parameter value.)
  • Compute the standard error of the sampling distribution. In a previous lesson, we showed that the standard error of the sample estimate of a proportion σ_P is:

    σ_P = sqrt[ P * ( 1 - P ) / n ]

    where P is the true population proportion and n is the sample size. Therefore,

    σ_P = sqrt[ ( 0.75 * 0.25 ) / 100 ] = 0.0433

  Following these steps, we can assess the probability that the sample proportion will be less than 0.734. This is easy to do, using the Normal Calculator. We enter the following values into the calculator: normal random variable = 0.734; mean = 0.75; and standard deviation = 0.0433. Given these inputs, we find that the cumulative probability is 0.356. This means that if the true population proportion is 0.75, then the probability that the sample proportion will be less than 0.734 is 0.356. Thus, the power of the test is 0.356, which means that the probability of making a Type II error is 1 - 0.356, which equals 0.644.