It’s a common dilemma in marketing research: You have 40 minutes worth of questions, but you know that your respondents only have about 20 minutes worth of attention, or less. The concern used to be that respondents would break off at about 20 minutes and not complete the questionnaire. However, research by SSI and others has shown that the real risk is that they will complete the survey, but with much lower quality of responses.
There are three main problems with longer surveys.
- The first is that all humans have “task unrelated thoughts” or TUTs. Basically, we are thinking about something other than what we’re doing. Ever arrived at work with no memory of driving there? TUTs in process. Some researchers have shown that the mismatch between what we’re doing and what we’re thinking about is as much as 30%. And guess what – the longer the task, the greater the mismatch.
- The second problem is technology. Surveys on mobile devices take longer to complete than the same survey on a laptop or desktop. So the same number of questions will not produce the same respondent experience. (Today, most mobile surveys are taken by respondents when they are at rest – at home, in the evening. The situation will get worse when respondents want to take surveys when they are actually mobile, in trains, planes or their self-driving vehicles.)
- And the final problem is that after about 20 minutes, human respondents tend to speed up their response time per question. So they are giving less attention to later questions than earlier questions. (This is known as “satisficing” – doing the job just well enough not to get kicked out of the study.)
So what can you do? The obvious answer is to limit your questionnaire length to no more than 20 minutes, and consider an even shorter questionnaire length for B2B surveys. But if your survey absolutely must take longer than 20 minutes, you can also improve respondent attention by:
- Use fewer characters per question. Shorter questions make the survey feel shorter than the same number of questions with more characters. (This might be another application of the KISS principle: Keep It Simple, Stupid!)
- If you expect respondents to use mobile devices to complete the survey, make sure the survey software is mobile-optimized. Not just mobile friendly, but truly optimized to give the mobile respondent the best possible experience.
- Consider whether all respondents have to answer all the questions. Using a split sample technique – where only part of the sample answers part of the questions – can shorten the average survey length significantly.
- Consider whether you have to ask all the questions. If you have asked the question before and the data has been stable over time, do you really need to ask that question? Or if you can pull information by respondent from your CRM or your panel vendor, can you avoid asking the question? For each question, ask yourself: “What are we going to do with it?” If it is not going to result in action, do you need to ask it? Or consider a factor analysis to pull out the most important factors in the survey.
SSI reports that their average survey length is 24 minutes; for Infosurv Research, our average survey length is between 10 and 12 minutes. So shorter surveys can be done! It takes managing your clients, focusing on what is most important for decision making, and discipline. Keep in mind: information that is nice to know is not information that you need to know!
If you haven’t already, read Part 1 and Part 2 of our B2B Research blogs
