Data Collection Methods

Before we can derive conclusions from data, we need to know how the data were collected; that is, we need to know the method(s) of data collection.

Methods of Data Collection

In this lesson, we will cover four methods of data collection.

• Census. A census is a study that obtains data from every member of a population. In most studies, a census is not practical, because of the cost and/or time required.
• Sample survey. A sample survey is a study that obtains data from a subset of a population, in order to estimate population attributes.
• Experiment. An experiment is a controlled study in which the researcher attempts to understand cause-and-effect relationships. The study is "controlled" in the sense that the researcher controls (1) how subjects are assigned to groups and (2) which treatments each group receives.

  In the analysis phase, the researcher compares group scores on some dependent variable. Based on the analysis, the researcher draws a conclusion about whether a treatment (independent variable) had a causal effect on the dependent variable.

• Observational study. Like experiments, observational studies attempt to understand cause-and-effect relationships. However, unlike experiments, the researcher is not able to control (1) how subjects are assigned to groups and/or (2) which treatments each group receives.

Data Collection Methods: Pros and Cons

Each method of data collection has advantages and disadvantages.

• Resources. When the population is large, a sample survey has a big resource advantage over a census. A well-designed sample survey can provide very precise estimates of population parameters - quicker, cheaper, and with less manpower than a census.
• Generalizability. Generalizability refers to the appropriateness of applying findings from a study to a larger population. Generalizability requires random selection. If participants in a study are randomly selected from a larger population, it is appropriate to generalize study results to the larger population; if not, it is not appropriate to generalize.

  Observational studies do not feature random selection; so generalizing from the results of an observational study to a larger population can be a problem.

• Causal inference. Cause-and-effect relationships can be teased out when subjects are randomly assigned to groups. Therefore, experiments, which allow the researcher to control assignment of subjects to treatment groups, are the best method for investigating causal relationships.

Test Your Understanding

Problem

Which of the following statements are true?

I. A sample survey is a type of experiment.
II. An observational study requires fewer resources than an experiment.
III. The best method for investigating causal relationships is an observational study.

(A) I only
(B) II only
(C) III only
(D) All of the above.
(E) None of the above.

Solution

The correct answer is (E). Unlike an experiment, a sample survey does not require the researcher to assign treatments to survey respondents. Therefore, a sample survey is not necessarily an experiment. A sample survey could be an observational study, rather than an experiment. An observational study may or may not require fewer resources (time, money, manpower) than an experiment. The best method for investigating causal relationships is an experiment - not an observational study - because an experiment features randomized assignment of subjects to treatment groups.