Survey Sampling

Survey Sampling

In statistics, survey sampling' describes the process of selecting a sample of elements from a target population in order to conduct a survey. A survey may refer to many different types or techniques of observation, but in the context of survey sampling it most often involves a questionnaire used to measure the characteristics and/or attitudes of people. Different ways of contacting members of a sample once they have been selected is the subject of survey data collection. The purpose of sampling is to reduce the cost and/or the amount of work that it would take to survey the entire target population. A survey that measures the entire target population is called a census.

Survey samples can be broadly divided into two types: probability samples and non-probability samples. Only surveys based on a probability samples can be used to create mathematically sound statistical inferences about a larger target population. Inferences from probability-based surveys may still suffer from many types of bias.

Surveys that are not based on probability sampling have no way of measuring their bias or sampling error. Surveys based on non-probability samples are not externally valid. They can only be said to be representative of the people that have actually completed the survey.

Put another way, if a probability-based survey of the United States household population finds that 59% of its respondents support a piece of legislation there is mathematical reason to believe that the proportion of all the persons living in households in the United States who support this piece of legislation is close to 59% (within the margin of error). If a non-probability survey conducted in the United States finds that 59% percent of its respondents support a piece of legislation that is the only conclusion that can be drawn, no statement about the target population can be made.

In academic and government survey research probability sampling is often regarded a standard procedure that must be employed regardless of the cost. The Office of Management and Budget's List of Standards for Statistical Surveys states that federally funded surveys must be performed,

selecting samples using generally accepted statistical methods (e.g., probabilistic methods that can provide estimates of sampling error). Any use of nonprobability sampling methods (e.g., cut-off or model-based samples) must be justified statistically and be able to measure estimation error.

Many statisticians disagree with these views. For example, Valliant, Dorfman and Royall explain,

To claim that, in general, probabilistic inferences are not valid when the randomization distribution is not available is simply wrong. This is not to deny that randomization is valuable, but only to deny that it represents the basis for all valid, rigorous, probabilistic inference.

The extreme position, that no inferences can be made unless the selection probabilities of the sample units are known would make it impossible to draw inferences from most samples. For example, most surveys have substantial amounts of nonresponse. Even though the units are initially chosen with known probabilities, the nonresponse mechanism is unknown and must be modeled, as in an observational study.