In marketing and survey research, sampling error occurs because we measure components of a sample of the population instead of the whole population. For example if you measured the average height of a basketball team, it would be reasonable to measure each player and get an accurate average height. However, if you wish to measure the average height of all the people in the city of Atlanta, it would be unreasonable (and cost prohibitive) to expect to measure everyone. So, you would take a random sample of people, measure their heights, and take an average as an estimate of the population. Because you are taking a sample, your measurement has some probability of being inaccurate.
Fortunately, thanks to statistical theory, it is possible to measure the probability and amount of sampling error for a given population and sample size. The size of the sampling error is determined primarily by the size of the sample that is taken from the population. The larger the sample that is taken, the smaller the amount of sampling error that is present in the estimations.
The metric we use to evaluate the amount of sampling error is called the margin of error and is typically stated as a plus or minus from the estimated statistic for a desired confidence level, usually a 95%. This confidence level is the probability that the margin of error around the estimated statistic would include the “true” percentage.
For example, if you surveyed 250 people out of a population of 1 million, you would have an estimated sampling error of +/- 6.2% with a 95% confidence. If you found that 50% of this sample were aware of your brand, this means that you are 95% certain that the actual percentage of the population aware would be between 43.8% and 56.2%.
In marketing research, we typically report a single margin of error for the maximum margin of error for an estimate of 50%. If the estimated percentages are larger or smaller 50%, the sampling error interval for the percentage will be somewhat smaller. Thus, using the example above, if the estimate of awareness of brand is 80%, then you would have a sampling error of +/- 4.9% with a 95% confidence. Similarly, as the response gets smaller for subsets of respondents answering individual questions, the margin of error becomes larger.
A common misperception in marketing research is that you have to sample a certain percentage (often 10%) of the population. Not so: the population size does not have an impact on the sample needed (or the related margin of error) unless the sample size is greater than 5% of the population. For example, for a sample of 250, if the population is greater than 5,000, the margin of error works as is. If the population is smaller than 5,000, you would need to add a correction factor onto the margin of error calculation.
So, you may say that all you need to do is simply use a very large sample to minimize your margin of error. However, there is a cost associated with sampling. Therefore, you need to balance the amount of sampling error that you can tolerate in your decision making with the cost of the sampling. For example, let’s assume that you are measuring the awareness of your brand to determine how much to spend on a new ad campaign. If your awareness is found to be below 60%, you will decide to implement the new campaign costing $250,000. A survey with a sample of 300 interviews, yielding a margin of error of +/- 5.6% at the 95% level of confidence will cost approximately $15,000. To lower the margin of error to under +/-4% would require a sample of 600, doubling the cost of the project. The sample size of 300 would probably provide you with the level of accuracy needed to make a confident and sound decision without extending budget.
The bottom line is that sampling error in marketing research is just a fact of the researcher’s life and another factor you need to manage. You can still accurately estimate the variance and present your findings appropriately, because sampling error is measurable. There are many, many sources of error that are less measurable and controllable, but that’s a topic for another day!