Published November 2006 in Marketing News, pg. 25-26
I remember an event from childhood when my mother asked my father to hire a plumber to fix a leaky faucet. “No, honey, that’s silly,” he retorted. “Why pay a plumber when I can fix the leak myself? Help me find my wrench.”
A few hours later we still had a leaky faucet—plus a puddle of dirty water, a broken cabinet door and an enraged mother.
In the home as well as the office, we constantly face the decision as to what to outsource and what to do ourselves. In today’s business environment of slashed budgets and tight corporate resources, doing “more with less” is the mantra. In business schools they teach that firms can maximize their efficiency by focusing on their core competencies and outsourcing their not-so-core competencies. This is a nice strategy for organizations with plenty of cash lying around, but what about the rest?
In the world of online research, the question of in-sourcing vs. outsourcing is always on our clients’ minds and often comes to us in the form of a common question, “Why should we pay you to conduct a survey online when we could do it ourselves using survey software for thousands of dollars less?” It’s a good question, and the answer is that sometimes they shouldn’t hire us.
There are many survey projects that are so simple it would be a waste of money to hire a full-service online research firm like us. With even a basic level of technical proficiency, many individuals can learn to create and execute surveys using any of the dozens of new software or ASP-based survey tools out there. With the convenience and familiarity of an online interface and more user friendliness than ever before, these tools make the technical side of online survey creation a snap. They also cost only a fraction of what one might pay to retain a full-service research firm.
Online survey software is well-suited for “quick and dirty” type surveys where only directional quantitative or qualitative data is needed. These surveys often run short, with 20 or fewer items, and do not require the statistical accuracy that a full-service research firm can assure. An example may include a survey of 20 PTA parents to see what they would like the school to accomplish that year for their children, where only a rough tally of program preferences is required. Such surveys also do not require complex response validation, piping or branching logic. They are simple, straightforward, and the survey creator knows exactly what questions to ask.
Survey software or online survey tools can also be used if the individual conducting the survey is proficient in questionnaire development, research methodology design, sampling and statistical analysis. This individual must also have a certain level of computer proficiency in order to properly use the tool they have purchased. For clients who have this level of expertise, the services of a full-service research firm are often redundant and unnecessary.
What one plans to do with the survey results should also be considered. If the results will only be used for internal purposes, never to be presented to investors, customers or the general public, the credibility that a full-service research firm lends is probably not needed. It also depends upon the magnitude and potential consequences of the decisions to be influenced by the survey results. If no major business decisions will be influenced by the survey data, the stakes are low and an in-house solution is probably all that is needed.
We encourage clients to bear in mind, however, that no survey tool can help them avoid leading or biasing questions in their questionnaire. Nor can it advise them when to use open-ended text response fields to collect rich verbatim comments vs. closed-ended fields to collect statistically valid data. It can’t tell them what sample size they need for true statistical validity, nor whether to provide an incentive to survey respondents. Though it certainly can produce pretty bar chart reports, it cannot interpret the reports and provide conclusions and recommendations that can be acted upon.
Survey software also doesn’t have access to respondent sample, nor can it advise the sampling frame what should be targeted in the first place. It can’t provide the project management of a survey project either, which is a step that many clients underestimate in importance and complexity. A full-service research firm provides the structure around a research project, which is vital to its success. This includes establishing deadlines and accountability, keeping the project moving through an established process and assuring that results are credible and thus more likely to be acted upon.
In short, survey software can always be used to conduct online research, but it can’t always be trusted to conduct online research right.
Every year, a portion of the prospects that come to us end up conducting their survey in-house using one of the survey software packages out there. This is fine for those potential clients who have in-house online research expertise, but worrisome for those who do not. We are concerned that many organizations do not know how to properly design a questionnaire, invite and incentivize respondents, and interpret results. Sometimes their questionnaires contain leading questions, poor wording, illogical flow and an inappropriate length, all of which can contribute to poor survey response rate and unreliable survey results.
There have been cases when, after researching full-service online survey research firms and their cost, a prospect decides to conduct their survey in house using a purchased survey software tool. Often, this prospective client comes back to the same research firms the next year practically begging for forgiveness. They learn the hard way that online research is not as easy as it looks.
Many individuals in the corporate world still believe if they can install survey software, they have all they need to collect accurate feedback from their employees, customers or target market. This is akin to believing that if one knows how to use a knife, he is ready to perform surgery. Of course, not all operations require surgical precision. The trick is to understand whether one needs to remove a splinter or a brain tumor. Most of us are fine removing a splinter with some tweezers and a few minutes of instruction but wouldn’t dream of performing brain surgery.
In the end, online survey tools certainly have their place. They have empowered millions to collect feedback more quickly and inexpensively than was ever possible in the past. However, these powerful data collection devices should be recognized for what they are: tools, not necessarily solutions.
We must always remember that survey software cannot turn a layman into a researcher, any more than a wrench could turn my father into a plumber.
Jared Heyman is founder and president of Atlanta-based Infosurv Inc., a firm specializing in online survey research.
This article originally appeared in the Nov. 1, 2006 issue of Marketing News. Copyright 2006 by the American Marketing Association. All rights reserved.