The Allure of Mobile Market Research

As UK blogger Ray Poynter is fond of saying, “Mobile has been the next big thing in market research for the last 15 years.” Hyperbole or not, the rapid adoption of mobile phones, especially smartphones, has opened up new avenues for researchers with the promise of expanding and enriching data collection beyond anything possible with current methods. For example:

The tendency for people to take their phones everywhere has created the potential for “in-the-moment” research – the opportunity to interact with people when they are making a purchase or experiencing a service.

The rapid spread of mobile telephone services in emerging markets now makes it possible to do research with populations formerly accessible only by personal visit.

The increasing power of smartphones with cameras and recording capabilities has deepened and enriched the kinds of information we can collect, especially in qualitative research.

The Current Reality

While these mobile research methods have begun to grow rapidly, they have yet to account for a significant percentage of the roughly $34 billion that clients spend worldwide on market research each year. Research agencies, especially those with significant investments in more traditional methods, have been slow to bring mobile applications to their clients. And many of those clients remain reluctant to embrace mobile methods in a major way, waiting for their value and validity to be widely accepted.

But respondents are not waiting and therein lies a major challenge. A surprising number have begun to use mobile devices – especially smartphones – rather than laptops or desktop computers to respond to online survey invitations. In 2011, Market Strategies International began monitoring the type of device its online respondents were using to start online surveys. We found that as many as 30 percent of invitees were starting some surveys on a mobile device, either a smartphone or a tablet. Other researchers in the U.S. and Europe have reported similar trends. We also found that the highest incidence of mobile responders occurred with samples drawn from customer lists, rather than from online panels.

Across the industry, researchers have begun referring to people using a mobile device when responding to an online survey invitation as Unintentional Mobile Respondents, or UMRs.

Why is this a problem?

Online surveys generally are not designed to be completed on a mobile device, especially the very small screen of a smartphone. They are intended for desktop or laptop completion and simply do not work well on a smartphone. Our data show that UMRs were twice as likely as other respondents to break-off the survey without completing it, and those who did complete took anywhere from 25 to 50 percent longer to do so. Those who completed on a smartphone were more likely to be younger (under 35), female and non-white. It is probably safe to assume that those who did not complete were demographically similar. In addition, they tended to provide less detail in open ends and to answer slightly more positively on satisfaction questions.

We tried intercepting these UMRs, suggesting they complete the survey on a computer instead, but few complied.

What should we do about it?

To state the obvious, it is essential that we find a way to create a mobile survey experience that reduces break-offs, takes no longer to complete than on a computer and minimizes any bias that may result due to the use of a mobile device.

The easy part is to change how the questionnaire is formatted on a smartphone so that respondents have a better experience. Creating two separate versions of every survey, one for mobile and one for computer, seems like the wrong approach given the extensive research record showing that different presentations of survey questions can produce different answers. On a practical level, producing two distinct versions of the questionnaire on every study is likely operationally cumbersome and expensive.

The optimal solution is a “mobile friendly” interface that creates a similar experience for the respondent regardless of the device (smartphone, tablet or desktop computer) and can be quickly and easily implemented. To that end, Market Strategies conducted a controlled experiment that tested seven different survey presentations: five on mobile devices and two on desktop computers. Based on this research, we recommend the two formats in the image.

Compared to most traditional designs developed for display on a PC, this mobile version uses larger fonts and interface elements. It is completely readable, whether rendered in portrait mode on a smartphone or in landscape mode on a large tablet. And compared to other so-called ”mobile friendly” designs that use sliders, dropdowns or other interactive elements that can be difficult to manipulate on a small screen, the design emphasizes direct input – minimizing the number of touches (or clicks) necessary to provide a response. Our research found that this design significantly reduced survey length and break-offs for mobile survey takers while improving their self-reported survey experience. Perhaps most importantly, this design yielded essentially identical response distributions for respondents regardless of the device – mobile or PC – that they used.

That is the easy part. The hard part goes back to questionnaire design. There is a longstanding consensus that for mobile research to succeed on a broad scale, questionnaires must be shorter, less verbose and contain fewer complex survey tasks. From the outset, questionnaire designers should assume that a significant number of respondents will complete the questionnaire using a small-screen mobile device and develop a questionnaire that will work well within that constraint. This is a major challenge, but one that respondents are telling researchers we must overcome if we are to meet them on their terms. It is non-negotiable.