Many marketing researchers believe that interpreting the resulting survey data is the most fun and interesting part of the job. Reviewing the data for the first time, trying to unlock the secrets and tease out the contradictions, and finally creating the meaningful story the data reveals is simply an intellectually fun and challenging task.
First, Go Back to the Beginning
The first step, of course, is to go back to the research objectives and the marketing/ business problem the research is meant to illuminate or solve. Understanding what challenge the data should address and what hypotheses were tested provides the foundation for your analysis. Typically, those require three types of approaches to data analysis and interpretation, as described by Chad Giddings, of Target Marketing Magazine: “Understanding the intended action that will come from information and its analysis is a good place to start. In rather simple terms, data and analytics are used for three broad purposes:
- Describe: Data can tell a story about what has happened. It is often a snapshot in time that provides information on—or a description of—something that has occurred.
- Predict: Data are often used to forecast a future occurrence or outcome. There are a host of predictive techniques that use sets of data points to make educated assumptions about the future.
- Decide: Data are almost always used to help inform and guide some business decision and action. Data are at the heart of most organizations’ performance measurement systems and are used to make key decisions.”
The second step is to go back to the methodology and understand what you are dealing with regarding data quality. How big is your sample? How large is the total population? How representative is your sample relative to the population? Do you have any clear and obvious biases (e.g., non-response bias) that might color your interpretation? Has anything happened in the business or social environment that could have influenced how people answered the survey? For example, a major competitive announcement of a new product launch, a price cut, or an acquisition could directly influence potential customers’ attitudes and perceptions of your business.
There are three common mistakes in data interpretation that are often the downfall of marketing research:
- Is it different? Statistically significant differences are easy to identify in marketing research results. Most analysis software includes this type of analysis, so you can quickly produce tests of significance for all sorts of data, question types, and research objectives. However, all significant differences are not created equal. If the survey software tells you that differences are significant, you still need to ask yourself two more questions:
- Is it big? If you are lucky enough to have a large sample size, some significant differences may appear on rather small numeric differences. For example, a sample of 1,000 might produce significant differences on proportions that vary by as little as 3 percentage points. In interpreting the data, it is not enough to know that an equation says the difference is significant. Is the difference large enough to be important for the business?
- Can we do anything about it? The key here is to inspire action. Every significant difference cannot be a business driver. Ask yourself, so what? If we know that these two data points are significantly different, and the difference is large, what can we do about it? Can we change our business in some way to take advantage of this or is it just “nice to know”?
- What was the question? Understanding how the question was asked will lead to different interpretations. For example, if the question was multiple-choice, is it important to report how many respondents gave each answer? Or would it be more insightful to consider which respondents selected the same choices? Moreover, if one answer is selected by the most respondents, does that mean it is preferred the most or simply table stakes? How do you present the results of a rank order question? Is the response selected most often the best? What if the responses were pretty much evenly distributed? Should you code the responses to a verbatim question so you can analyze them quantitatively? Or is it better to use the information for illustration and clarification of other findings?
- Qualitative vs. Quantitative? The old saw in marketing research is that quantitative data tells you “How many?” and qualitative data tells you “Why?” While that is probably an overgeneralization, combining qualitative and quantitative information can lead to much richer and more insightful data interpretation.
Answer the Questions
One of the biggest complaints that marketing researchers hear is that we “dump data” on our audiences. Keep in mind that not everyone processes data quickly or easily. Whether in producing a physical report or in presenting your findings, keep the jargon and detail to the minimum needed to support your findings. Make sure your report meets the project objectives – or explain why it doesn’t. Don’t hide behind the research – take a stand and tell your audience what you think the findings mean for their decisions. And, of course, focus on action. Answer the question: What should we do?
Marketing research that is correctly interpreted is a valuable tool that can change business operations and results. Interpreting the data appropriately and correctly is key in providing information and direction that executives can rely on to drive success. The responsibility for the correct interpretation is fundamental to the success of any marketing research project.