Notes from brainstorming sessions at the 2005 CMOR Respondent Cooperation Workshop in Washington, DC.
1. Develop ideas on ways to overcome the increasing non-contact rates in phone surveys. How do you decrease non-contact and obstacles to getting through to respondents? What are other methods that could be tried?
- Recognize people who are over-sampled.
- Send advance letters that are personalized.
- Flexibility with introductions.
- Incentives - $2.
- Identify sponsor.
- Educating the public on who we are.
- Improve the image of market research.
- Sample Management.
- Targeting specific times (e.g. elderly stop at 7:00).
- Multiple call attempts.
- Mixed Methodology.
- Using Voicemail.
2. Develop ideas on ways to overcome the increasing non-contact rates in mail surveys. How do you decrease non-contact and obstacles to getting through to respondents? What are other methods that could be tried?
- New $1 Bills.
- Specialized Fist Class Stamps (chosen for audience) on outgoing and reply envelopes.
- Personalized letters signed in blue ink by a meaningful person.
- Small booklets rather than 8 ½ x 11.
- Phone reminder to encourage response via mail and to allow for phone response.
- If budget allows use Priority Mail or FedEx.
- Pre-contact by phone and or by mail to encourage respondent.
- Letter sent with DVD or CD about topic and advantage – The value of research.
- Scented envelope or questionnaire.
- Provide an 800 number on the questionnaire and letters to answer the respondent’s questions.
- Insert chip that plays music or a voice message.
- Questionnaire and letters in English and Spanish.
- Insert a pen or pencil in the envelope.
- Address for response should be on questionnaire in case envelope is lost.
- Odd shaped questionnaire (circle, oval, etc.) if it relates to subject or sponsor.
- Pop-up questionnaire.
3. Develop ideas on ways to overcome the increasing non-contact rates in internet surveys. How do you decrease non-contact and obstacles to getting through to respondents? What are other methods that could be tried?
- Recruit by phone and email the link to respondents – simultaneously. 2 week follow-up by phone.
- Big money incentives.
- Draw from panel.
- Specially-trained interviewers to bypass gatekeeper and leave message either on voicemail or with secretary.
- Shorter survey.
- Allow respondent to save partially complete and then return to it later.
- Immediate incentives at the end of the survey (e.g. EBay, e-rewards, ring-tones, gift cards, I-tunes, etc. relevant products).
- Ensure that target population is internet savvy and survey is appropriate for internet administration.
- Use multi-model recruiting (not just internet and, or email) with something like snail mail.
- Administer marketing and recruiting strategies in multiple ways, phases, and times.
4. Develop suggestions to increase internet survey response rates. What steps could be built into internet methodology that would lead to higher response?
- Send introductory letter.
- Offer incentive.
- Improve instrument design.
- Keep survey short.
- Clear log on instructions explaining (subtly) web address vs. search.
- Email invitation.
- Make an agreement with an internet accessible vendor for use by non-connected respondents.
- Phone, mail, or email reminders.
- Include an 800 number in the invite for questions.
- Offer alternative completion methods.
- Offer charitable donation alternative.
- Keep technology accessible to lower end systems.
- Make screens with simplified low graphic designs.
- Keep formatting consistent.
CMOR is already working on a Research Industry Identifier. Discuss and develop other methods the research industry could use to get the message out on the value and importance of doing surveys and focus groups. If anyone in the group has done awareness or publicity for research, describe what was done and how well it worked.
- Paid advertising – make public more aware.
- Visual media.
- Find most cost-effective media sources.
- Link advertisement to relevant segment of the program – i.e. Frank Newport.
- Add logo to any letters, media, or poll results.
- Media plan needs to address DNC and SPAM.
- Banner on well-known and highly used web sites.
- Paid link on appropriate search topics.
- Billboards in appropriate places.
6. Discuss the pros and cons of using panels for surveys. What are the concerns you have about using people recruited as professional respondents?
- Part A:
Refusal rate is lower.
Panelist dissatisfaction with process.
Call backs are less difficult.
- Part B:
- They put a lot more thought into their responses (this could be bad or good).
- Respondent holds onto details because they know they will be called (good or bad).
- Lack of top of the mind responses.
7. What are the obstacles to converting a telephone center to completely bilingual interviewing? What steps would management need to take to create such a center? What are the concerns in managing bilingual interviewers?
- Finding sufficient and consistent client and or project volume to warrant expense.
- Finding qualified, dependable, and reliable staff.
- Finding fluency in both languages.
- Increase project volume.
- Determine best market location for facility.
- Create a bilingual division from the ground up.
- Hiring qualified staff and supervisory personnel.
- Reliability in scheduling and adherence to the schedules.
- Quality control.
8. What are some ideas your companies have used for incentives on surveys? Discuss how effective those incentives were for increasing response. What incentives did not seems to work?
- Small cash incentives (before and during) and pre survey mailer for certain groups.
- Small cash incentives after the survey or to avoid a terminated call.
- Try the product and keep it.
- Larger cash incentives business-to-business (health) amounts depending on access and expectations.
- Client recognition in the beginning of the survey.
9. Can you interview the public without incentives? Why or why not? Defend your position.
- Yes – We already do, it has been proven. Depends on the length and what is being asked for example: fill out something, look through cards require money, while public opinion can be argued for the public good. Many people find political polls interesting and like to hear the questions and see results, “Hey, that’s the poll I participated in.”
- The amount of exposure to surveys as a respondent. Is the question, “should we?” Analogy: you wouldn’t not pay your interviewers, since they are your livelihood, so why not pay the interviewers livelihood?
10. Develop suggestions of mixed mode surveys that would work well with:
- Young people 18-34.
- Minorities and Ethnic groups.
- Affluent households.
From what you heard this morning, and from your own experience, what combinations in what order would work best with each group?
- Young people 18-34: Web, phone, and internet.
- Minorities and Ethnic groups: Phone in language, mail in language. Target areas – not by name.
- Affluent households: Mail and phone – targeted sample.
11. In doing telephone surveys, what pitfalls and obstacles has your organization encountered in improving and maintaining response rates? What has your firm done to reverse the downward trend?
- Bad telephone numbers – Research to find better numbers.
- Caller ID – Clever name or moniker.
- Respondent tries to schedule callback and because it’s RDD, the caller cannot schedule it.
- Hearing impaired respondents.
- Respondents with answering machines (vs. voice mail) screen their calls.
- Offers of pre and post incentives to reverse downward trends.
12. In doing mail surveys, what pitfalls and obstacles has your organization encountered in improving and maintaining response rates? What has your firm done to reverse the downward trend?
- Ignore it.
- Language (mail in English).
- Good addresses.
- Incomplete returns.
- Literacy issues.
- Translation into multiple languages.
- Research where to find respondents:
- In the field.
- In the databases.
- Locating respondents:
- Individuals. Try more steps.
- Tracking through contacts.
- Easy to follow.
- Advance letters and materials.
- Pre-testing of questionnaires.
- Shorten length.
13. What effect has your company seen from the Do Not Call laws and the decrease in telemarketing calls? Describe the responses and techniques you have used to overcome objections to calls on the basis of the new laws.
- I want to reassure you that the DNC list does work. It is designed for telemarketers who want to sell you something. We are calling to get your opinions…
- Voice tone is essential when responding to this objection. The interviewer needs to be calm and reassuring.
- The job of the interviewer in this case is to educate the respondent and convert if possible, but not expect it.
14. What aspects of phone surveys have you found in your projects decrease response rates or produce bias? How did you determine that aspect of the survey was the source of the decline or confusion? What did you do about it?
Caller ID and Privacy Manager.
800 Info Number (educate respondents).
Questionnaire (subject, wording, lengthy).
Better, Shorter questions – Interviewer skills.
15. What do you think CMOR should work on in your behalf to help response rates in the industry?
- Ad campaign to inform the public why participation in surveys is important. (PSA) – Multi-lingual.
- Reach out to segments of the research community that have expertise in emerging populations.
- Explain to the public what organizations belong and do not belong on the Do Not Call list.
- CMOR “Seal of Approval” on surveys.
- On-line workshops for front line employees – interviewer and supervisor training.
- Emphasize that the same quality control procedures used in English surveys apply to non-English surveys.
16. What is your organization doing to measure non-response rates? What evidence do you have?
- Tracking different kinds of non-response.
- Interviewer observation of interaction with household (use categories code) – (used to run reports).
- Analysis of time and date of non-response.
- Offer a shorter version of questionnaire for those that were a non-response.
- Money – the more you offer, the more likely you are to turn around a non-response.
- Focus groups prior to interviewing (ask focus groups concerns and what to say to respondents).
17. List in priority order the most significant challenges and obstacles your firm faces today in developing supervisors and training them. What suggestions do you have to overcome those obstacles?
Obstacles (listed in order):
- Shifts – Balancing: when to bring them in before a shift for preparation and how long to keep them after the shift for reports.
- Consistency among supervisors – styles and personality.
- Appropriate people for the job – experience in market research versus call center versus general supervisory.
- Over burdening the supervisors – keep them focused on the key priorities.
- Step training.
- Consistent training for the supervisors.
- Policies and procedures manuals (for consistency).
- Promoting from within.
18. How do you identify good supervisory candidates? What are some of the methods your organization uses to confirm that interviewers have the skills needed to do the supervisor job?
- Good people Skills.
- Previous interviewing experience.
- Good interviewing skills.
- Good organizational skills.
- Attention to detail.
- Flexibility: adapting to different technology and projects.
- Ability to motivate.
- Critical thinking and good decision-making.
- Behavioral interviewing (asking the candidate, “what would you do if…?”).
- Varied experience and surveys.
19. Describe your current training program for supervisors. Compile a non-duplicate list of all aspects each of your companies use. Then, what aspects or components would you like to see developed for companies to use?
- Task training specific to supervising project (interviewing training if needed).
- Leadership and team building training.
- Diversity training.
- Conflict resolution.
- Employee Development (individual development plans and performance improvement plans).
- Feedback and Communication (how to give, how to measure results, and monitor future performance).
- Project management (administrative, i.e. scheduling).
- HR policy (how to interpret, promote, and correct behavior).
20. How do you recognize supervisory burnout on the job? What are some ways to prevent it from happening? What are some ways to eliminate it once it occurs?
- Change in personality.
- Increased questioning and, or complaining.
- Lower quality of performance.
- Ongoing development.
- Variety of duties.
- Increased responsibility – high profile duties.
- Find out their needs and try to meet them.
- Move to other department if possible.
21. Is your organization using ART?
If Yes: What obstacles have you run into while implementing ART, and how have you overcome them?
If No: What is preventing your organization from adopting ART into your program?
- Yes, some – varies among companies represented at our table.
- Obstacles: time, money, mode, type of respondent, and type of study.
22. How does your organization measure training effectiveness? For interviewers and for Supervisors? If you have no measurement in place, how would you go about implementing a measure? What would it consist of?
- Role plays and mock surveys.
- Jeopardy game to test learning.
- Quizzes on human subjects training.
- Calibration exercise (joint monitoring session).
23. For the following types of research, what kind of research training (interviewer or Supervisor) and focus should CMOR work on?
- Mall Intercept.
- Focus Group.
24. Discuss your supervisory evaluation process. What is included in the evaluation (what is rated or assessed) how often do you evaluate supervisors, and who does the evaluation?
- HR – “sanctioned” supervisory competency list with guideline book: i.e. Critical Thinking, Organization, Communication, and Project Management, etc.
- Annual Review.
- Formal 6 month review.
- Quarterly – check pts (feed into IDPs [Individual Development Plan]).
- Direct Report – skip level (down).
25. How do you convince your clients that the time and money spent on training interviewers and supervisors to improve respondent cooperation is worth it to their bottom line and to their data?
- Sales point for contracts – we offer ART. What that means to client (PRR, quality, decreased attrition).
- Premise is wrong that training costs.
- Cost efficient.
- Use less sample.
- Less interviewer hours used.
- Interviewer better at representing your company.