As someone who is a past MRA leader and has been a MR professional for more than 35 years, people occasionally come to me with questions about MRA and the MR profession. Usually the questioner is someone who I may have met at a conference, was referred to me by someone else, or found my name through a Web search.
The question(s) I am asked are for the most part easily answered, but occasionally the question might deal with an aspect of MR in which I lack experience and expertise. When this happens, I volunteer to try to get them their answer. I do this for a couple of reasons. First I know how frustrating it can be for someone seeking an answer to get passed along from one person to another trying to get an answer. Second, at this point, I am usually interested in learning the answer myself and if I just pass the questioner to someone else, I fail to learn and don’t satisfy my curiosity.
Recently, I was called by someone who not only posed a question outside of my area of expertise, but which was so unusual, it had never even been posed to the practitioners who I called. It dealt with CATI interviewing.
The question I was asked was, “Is there an ‘industry’ accepted standard for recordation errors for CATI interviews?” What the questioner was asking about was specifically to what degree does the interviewer make a mistake when recording the answer the respondent provided? This is a more unusual question then you might think. Most of the telephone interviewing facilities I spoke with concentrate on quality control during the interview by monitoring the audio and only occasionally both the audio and video. They usually actively police their interview staff with regard to interview length (to prevent abuse), and they train their interviewers not to lead the respondent or interpret the answers they are provided. However none of those I spoke with measure recordation errors and have come up with the rate at which they occur. Several of them even questioned how they would measure this.
I told the questioner that I couldn’t imagine it would exceed one-two percent based on my knowledge of data entry errors from paper surveys, and the interviewing companies I spoke with thought that that was probably a reasonable answer.
As researchers, we understand that there is an error factor associated with research. Although we frequently only report the error rate associated with the sample size, error is cumulative and can occur at many stages in the process. We need to always be alert to all the places errors occur so that we can minimize their impact on our work.
Do you ever get asked unusual research questions?
What is your opinion about the question I was asked?
Photo credit: Toshiba Laptop by garryknight