How Probability and Non-Probability Samples Differ
Samples are an important part of market research since making direct observations of all members of a population under study would typically make for a very tough challenge.
A sample is a subset, or smaller group, within a population. When designing studies, it's critical to ensure that the sample replicates the larger population in all the characteristic ways that could be important to the study's research findings.
Some samples so closely represent the larger population that it's easy to make inferences about the larger population from your observations of the sample group. In market research, there are two general approaches to sampling: Probability sampling and non-probability sampling.
In the technique of probability sampling, also known as random sampling, everyone in the population has an equal chance of being chosen as a representative sample:
- Everyone in the sample must have the same probability, or fixed opportunity, to be in the sample set,
- AND the probability of any member of the sample group being selected for the sample can be mathematically calculated. In other words, everyone has the same, a fair chance of being selected.
The characteristics can be summarized as follows:
- Random basis of selection
- Fixed, known opportunity of selection
- Used for conclusive research
- Produces an unbiased result
- The method is objective
- Can make statistical inferences
- The hypothesis is tested
One of the most noteworthy features of the method of non-probability sampling, also known as non-random sampling, is that there isn't any specific probability that any given person will be in the sample set. In other words, you don't know which person from a population will get chosen for the sample.
Some characteristics of non-probability sampling include:
- Arbitrary basis of selection
- Used for exploratory research
- Produces a biased result
- Uses a subjective method
- Can make analytical inferences
- The hypothesis is generated
An Important Limitation of the Non-Probability Sampling Approach
An important limitation of non-probability sampling is that inferences cannot be drawn about the larger population based on a non-probability sample. This is not always the case, however, since a realistic view of how people approach research findings will readily identify situations where people do inappropriately draw conclusions from findings associated with non-probability samples.
Potential Sampling Errors
When working with non-probability samples, it is important to understand the occurrence of sampling error. The smaller the sampling group, the greater the chance of sampling error. One particular type of bias is a result of non-participation. It is important to understand the impact of non-participation on the overall outcome of a study.
For example, in the 1980 General Society Survey (GSS), those who did not participate in the research were found to be quite different, as a group, from those who had participated. The hard-to-reach group members were significantly different from their peer labor force participants—most markedly in socioeconomic status, marital status, age, the number of children, health, and sex.
Convenience samples are commonly used in social science and behavioral science because of the heavy reliance on college students, patients, paid volunteers, members of social networks or formal organizations, and even prisoners.
The purpose of much social science and behavioral science research is to verify that certain characteristics occur or do not occur in the group undergoing study. A common approach is to look for relationships among several attributes. Convenience samples are useful and adequate for this type of study, although a convenience sample is not always easy to put together.
Convenience samples may also be matched in order to compare two groups. In order to use matched convenience samples, a researcher must be able to identify a counterpart for each member of the first sample. These counterparts are members of the second (matched) sample.
The variables that are commonly matched include gender, age, race, ethnicity, educational attainment, place of residence, political orientation, religion, job type, and wages or salary. Matching these variables helps to reduce sources of bias, although it's important to recognize that even careful matching may not result in samples free of bias—there is always a possibility of bias from hidden sources.
Purposive sampling is used when the research design calls for a sample of people who exhibit particular attributes. Generally, these attributes are rare or unusual and are typically not distributed normally (according to the "normal curve") in the larger population. Purposive sampling is fraught with bias, some of which occurs as a result of the methods that are used to identify the members of a purposive sample.
For example, if the research purpose requires studying Veterans with traumatic brain injury (TBI), then the sample must consist of ex-members of the military who have sustained a traumatic brain injury, and who identify themselves accordingly and agree to participate in the study. Each of these attributes or conditions contributes a measure of bias to the sample, thereby limiting the level and type of conclusions that result from the study.
Samples that act like public opinion polls are disseminated with the idea that they represent how members of a population will vote in a coming election or the like. These samples must be highly representative of the population in order to be used to make forecasts about election results, for instance.