Survey Sampling
Sampling refers to the process of choosing a
sample of elements from a total
population of elements.
Probability vs. Non-Probability Sampling
Statisticians distinguish between two broad categories of sampling.
-
Probability sampling. With probability sampling, every element
of the population has a known probability of being included in the sample.
-
Non-probability sampling. With non-probability sampling, we
cannot specify the probability that each element will be included in the
sample.
Each approach has advantages and disadvantages. The main advantages of
non-probability sampling are convenience and cost. However, with
non-probability samples, we cannot make probability statements about our sample
statistics. For example, we cannot compute a
confidence interval for an estimation problem or a
region of acceptance for a hypothesis test.
Probability samples, in contrast, allow us to make probability statements about
sample statistics. We can estimate the extent to which a sample
statistic is likely to differ from a population
parameter. The remainder of this tutorial focuses on probability
sampling.
Quality of Survey Results
When researchers describe the quality of survey results, they may use one or
more of the following terms.
-
Accuracy. Accuracy refers to how close a sample
statistic is to a population
parameter. Thus, if you know that a sample mean is 99 and the true
population mean is 100, you can make a statement about the sample accuracy. For
example, you might say the sample mean is accurate to within 1 unit.
-
Precision. Precision refers to how close estimates from
different samples are to each other. For example, the
standard error is a measure of precision. When the standard error is
small, estimates from different samples will be close in value; and vice versa.
Precision is inversely related to standard error. When the standard error is
small, sample estimates are more precise; when the standard error is large,
sample estimates are less precise.
Margin of error. The margin of error expresses the maximum
expected difference between the true population parameter and a sample estimate of that
parameter. To be meaningful, the margin of error should be qualified by a probability
statement. For example, a pollster might report that 50% of voters will choose
the Democratic candidate. To indicate the quality of the survey result, the
pollster might add that the margin of error is +5%, with a
confidence level of 90%. This means that if the same sampling method were used to select different samples
and if a margin of error were computed for each sample,
the true percentage of Democratic voters would fall within the margin of error 90% of the time.
In upcoming lessons, we'll show how to measure precision and margin of error
with different sampling methods and with different population parameters (i.e., mean scores,
proportions, and total scores).