Within the sphere of marketing research activities today, few have lower prestige than the simple survey. If you read the blogosphere, you see questions asking whether primary marketing research will be dead in 10 years (or maybe 5), whether Big Data will eliminate the need for surveys completely, and whether traditional “data collection” has a future. I found this response particularly interesting in a world where virtually every informed person I know spends an increasing amount of time online doing research, swapping ideas with colleagues, and gathering information that affects all corners of one’s life.
Marketing research taking a beating
If marketing research is taking a beating, the one vehicle getting hit hardest is the simple survey – once the staple of all marketing research. There are an array of reasons for this, but four stand out:
- Vision – Several years ago, Stan Sthanunathan, Senior VP for Consumer and Market Insights at Unilever (@ssthanunathan) caused tremendous buzz with the observation that 80% of all marketing research was backward-looking whereas the important marketing and strategy issues all are forward-looking. Historically, surveys have been used to track past behavior and changes in behavior and attitudes over time – hardly a forward-looking view.
- Viability – When surveys are used to predict the future, results are spotty at best. Nate Silver (@NateSilver538) and his fivethirtyeight.com site have demonstrated over the past several election cycles that simple surveys are of questionable value in predicting elections. His approach, which aggregates surveys, identifies patterns of historical bias in each survey organization, and aggregates the results against historical predictors and secondary data, has fared far better at state and national levels. This is not merely a U.S. phenomenon; the recent U.K. elections that pollsters considered “too close to call” resulted in a rout for David Cameron’s Conservatives.
- Value – In the era of “Big Data,” we can usually track the past far more effectively using actual stakeholder behavior than survey results. Data today can link consumers to virtually everything they buy, physicians to the medications they prescribe, and individuals to the interests they display and conversations they join online.
- Variability of sample – Today, it is virtually impossible to recruit and deliver a random sample of respondents for a reasonable price. If the sample is not random, it becomes difficult to decide how much to trust the results of the survey.
So, if we take a forward-looking view of the world, why conduct surveys at all?
There are types of data you can collect more cost-effectively and reliably through a survey than through newer methods:
- Simulating the future. Gamification and interactive simulation are hot, high-tech ways to simulate how purchasers or decision-makers will respond to future events (new products, changing regulation, market dislocations). For B2B decisions, which might involve price lists, formularies and/or complex algorithms, a web-enabled, survey-style simulation may be the most reliable way to replicate future events.
- Executing experimental designs based on multivariate analytics. Techniques such as choice methods and MaxDiff rely on the visual interplay of attribute combinations or word pairings that work best in a simple visual setting.
- Filling in missing pieces. Efforts to predict purchase behavior at an individual level rely on an array of independent measures, including brand attribute ratings, demographics, and descriptions of critical goals and targets. A survey is often the most straightforward way to obtain these measures.
- Providing inputs for more complex models. The fivethirtyeight.com model I mentioned earlier relies on the entire range of election opinion polls as critical inputs to the model. Survey-derived probabilities often define key parameters for Monte Carlo and Bayesian models.
It isn’t so much that the age of the survey has passed, but it is clear that the uses and applications of that vehicle are in a state of evolution – right along with the rest of our industry.
In the book Chapters from my Autobiography, Mark Twain popularized what he claimed was a Benjamin Disraeli quote, “There are three kinds of lies: lies, damned lies, and statistics.” In this situation, though, a different Twain quote might apply: for surveys, “The report of my death was an exaggeration.”
What do you see for the future of survey research? How are you adapting the simple survey to feed forward-looking business needs?