In Part 1 of this two-part series, we talked about the problem of sample quality afflicting the marketing research industry. Using ESOMAR’s recent publication, we highlighted six tips that should result in better sample quality. Now, we continue the discussion with an additional seven tips.
- While we all know that no one gets rich by being a respondent in an online panel, there is no denying that incentives are a powerful motivator for many respondents. Just Google “get paid to take surveys” and review the 418 million results, ranging from $1 to $90 per survey! Ask your sample provider about incentives – not just what they will pay for your survey, but the range of payments (including points systems, merchandise, etc.) that they use overall with potential respondents.
- Data quality checks. Who is responsible for checking data quality? Are systems in place to catch flatliners, speeders, and other low-quality respondents? Who determines who gets removed from the sample – and who counts toward the completion quota? These are critically important to ensuring data quality, but often marketing researchers just “assume” that someone will take care of it for them!
- Are your respondents happy? Does your sample provider conduct respondent satisfaction surveys? What do they do about respondents who post negative reviews? Whether they simply remove the dissatisfied respondents from the sample or try to “rehabilitate” them, it is important that you understand their philosophy as well as the impact their policy potentially has on your data.
- Status Reports and Debriefs. What information is shared with the client after the project is over? In addition to being important to calculate an accurate response rate, this information will give you important hints about the quality of the sample you purchased – and relied on for your project. You should expect to see the gross sample, start rate, participation rate, drop-out rate, number of poor quality responses, and so on.
- Exhausted Participants. Each potential respondent is a valuable asset for a sample provider, but there needs to be a balance between surveying them to exhaustion and preserving them to provide excellent quality data. Ask your sample provider how many surveys a respondent can take in a specific period of time. When are respondents retired simply to keep the sample source fresh? Are they able to monitor respondents who participate in many competitive panels? How do they handle those “professional respondents”?
- Kiddie Research. While most marketing researchers will go their entire career without facing this situation, it is important to remember that children under certain ages (depending on country law) cannot be interviewed without parental permission. ESOMAR has a sound guideline on interviewing children and young people, but your vendor should also be conversant in local laws (such as COPPA in the United States).
As a final note, keep in mind that these questions relate solely to acquiring sample participants. If your sample company is also hosting your survey, there are other questions that you will need to ask. While we focus on sample right now to avoid “garbage in, garbage out,” the fact remains that all the pieces of the marketing research project must fit with the overall project objectives to have a successful outcome.
Concerned about sample quality? We are too. Contact Us and we can help!