So much Data, So Little Insight

Every article you can find about trends in marketing research includes technological advances that yield more and more data: artificial intelligence, automated online surveys, mobile, cloud computing, data analytics, the Internet of Things (IoT), and behavioral economics. According to Leonard Murphy, executive editor, and producer at GreenBook, and senior partner with Gen2 Advisors, these technologies are “disrupting an inefficient process,” enabling researchers to deliver faster and more cost-effective insight. That drive from marketers and brands to make marketing research cheaper and faster has the interesting side effect of producing even more data than previously.

However, all this data does not seem to be producing a commensurate increase in insight. The 2017 Q3-Q4 GreenBook Research Industry Trends (GRIT) Report indicates that the industry has a 70% customer satisfaction rate. In the GRIT Report, “customers” are any client paying for research products, which is frequently people within market research, as opposed to pure marketers. If only 70% of your most knowledgeable customers were satisfied, you would do something about it!

There is no sliver bullet to solve this industry-wide problem. Technological disruption leads to change, and of course, it may take some time for the industry to shake out, adapt to, and take advantage of this change. In the meantime, here are a few observations on what we will see in the next few years.

  1. A Dearth of Data Scientists. Scratch any marketing researcher and you will probably not find a data scientist. You might find a marketer, or a social scientist, or psychologist, with strong data analytical skills. However, that’s not a data scientist. A data scientist is, according to SAS, “Data scientists are a new breed of analytical data expert who has the technical skills to solve complex problems – and the curiosity to explore what problems need to be solved. They’re part mathematician, part computer scientist, and part trend-spotter.” Moreover, because there is a scarcity of data scientists, they are in high demand. Which makes the fact that many data scientists are dissatisfied with their jobs somewhat confounding. According to this article by the Financial Times, data scientists typically “spend 1–2 hours a week looking for a new job”. Furthermore, the article also states that “Machine learning specialists topped its list of developers who said they were looking for a new job, at 14%. Data scientists were a close second, at 13%.” This may be that job expectation, and job reality is not aligned, but whatever the reason, we need to address this because the industry is facing a severe lack of experienced data scientists.
  2. People, not transactions. An important reason that marketers and brands are not seeing the increase in valuable and important insights promised by this overload of data is that much of the information is collected starting from the point-of-view of transactions, rather than people or customers. Especially for brand and social marketing communities, this is a major problem.  As explained by Rachel Happe, writing in The Community Roundtable, “Most community and enterprise social networking platforms have databases designed to report on content and transactions. It is either time-consuming or impossible to access behavior and life-cycle metrics by individual or user segment. This creates a troubling situation where the blind are leading the blind. Community manager’s core skill set is in community engagement – not data or business analytics. This makes sense, but also means that they typically don’t have the skills to understand and evaluate the data they can access in the platforms. This also means that they do not ask vendors for what they really need either, so vendors carry on and give them more and more meaningless data. This vicious cycle makes it even harder for them to understand and report value back to their organizations.”
  3. Qualitative rules – again. If all this data isn’t creating more and stronger insights, then marketers and brands will turn to qualitative to understand the motivations and values driving customer behavior. So qualitative research should see an increasing role in helping marketers get a better picture on their customers, or what Murphy calls a focus on “consumer closeness”. According to Veda Konduru, “the best insights come from a combination of quantitative and qualitative research. They complement each other. And, if you want a fuller picture for further fine-tuning, you’ll have to keep them both in mind.” Some of the ways that qualitative research can enhance big data are to reconcile behavior with opinions and perceptions, to fine-tune prediction models, and to test hypotheses. However, most importantly, qualitative research allows us to continue the story presented by analyzing past behavior and to humanize the data.

There is no doubt that automation and technology will drive marketing research. If we want to drive insights instead, we’ll need to maintain a balance and monitor how these changes will affect our clients: the business and brand marketers. We need to continually keep in mind that truly actionable insight – not just lots of data – is the only result that will increase client satisfaction with marketing research.