How to Choose the Best Sampling Method

The best sampling method is the sampling method that most effectively meets the particular goals of the study in question.

The effectiveness of a sampling method depends on many factors. Because these factors interact in complex ways, the "best" sampling method is seldom obvious. Good researchers use the following strategy to identify the best sampling method.

• List the research goals (usually some combination of accuracy, precision, and/or cost).
• Identify potential sampling methods that might effectively achieve those goals.
• Test the ability of each method to achieve each goal.
• Choose the method that does the best job of achieving the goals.

The next section presents an example that illustrates this strategy.

How to Choose the Best Sampling Method

In this section, we illustrate how to choose the best sampling method by working through a sample problem. Here is the problem:

As noted earlier, finding the "best" sampling method is a four-step process. We work through each step below.

• List goals. This study has two main goals: (1) maximize precision and (2) stay within budget.
• Identify potential sampling methods. This tutorial has covered three basic sampling methods - simple random sampling, stratified sampling, and cluster sampling. In addition, we've described some variations on the basic methods (e.g., proportionate vs. disproportionate stratification, one-stage vs. two-stage cluster sampling, sampling with replacement vs. sampling without replacement).

  Because one of the main goals is to maximize precision, we can eliminate some of these alternatives. Sampling without replacement always provides equal or better precision than sampling with replacement, so we will focus only on sampling without replacement. Also, as long as the same clusters are sampled, one-stage cluster sampling always provides equal or better precision than two-stage cluster sampling, so we will focus only on one-stage cluster sampling. (Note: For cluster sampling in this example, the cost is the same whether we sample all students or only some students from a particular cluster; so in this example, two-stage sampling offers no cost advantage over one-stage sampling.)

  This leaves us with four potential sampling methods - simple random sampling, proportionate stratified sampling, disproportionate stratified sampling, and one-stage cluster sampling. Each of these uses sampling without replacement. Because of the need to maximize precision, we will use Neyman allocation with our disproportionate stratified sample.
• Test methods. A key part of the analysis is to test the ability of each potential sampling method to satisfy the research goals. Specifically, we will want to know the level of precision and the cost associated with each potential method. For our test, we use the standard error to measure precision. The smaller the standard error, the greater the precision.

  To avoid getting bogged down in the computational details of the analysis, we will use results from sample problems that have appeared in previous lessons. Those results are summarized in the table below. (To review the analyses that produced this output, click the "See analysis" links in the last column of the table.)

  Cost	Standard error	Sample size	Analytical details
  Simple random sampling
  $3600	1.66	36	See analysis
  Proportionate stratified sampling
  $3600	1.45	36	See analysis
  Disproportionate stratified sampling
  $3600	1.41	36	See analysis
  One-stage cluster sampling
  $3600	1.10	720	See analysis

  Because the budget is $3600 and because each test session costs $100 (for the proctor), there can be at most 36 test sessions. For the first three methods, students in the sample might come from 36 different schools, which would mean that each test session could have only one student. Thus, for simple random sampling and stratified sampling, the sample size might be only 36 students. For cluster sampling, in contrast, each of the 36 test sessions will have a full class of 20 students; so the sample size will be 36 * 20 = 720 students.
• Choose best method. In this example, the cost of each sampling method is identical, so none of the methods has an advantage on cost. However, the methods do differ with respect to precision (as measured by standard error). Cluster sampling provides the most precision (i.e., the smallest standard error); so cluster sampling is the best method.

Although cluster sampling was "best" in this example, it may not be the best solution in other situations. Other sampling methods may be best in other situations. Use the four-step process described above to determine which method is best in any situation.