Archive for September, 2020

There may be no wrong answers, but there are wrong questions: Avoiding pitfalls in qualitative interviewing

September 17, 2020 Leave a comment

We always tell our participants that there are no wrong answers, and it’s true. They are the experts, and if we’re not learning what we want from them, the blame is likely on us. Maybe our goals or our expectations for the research are misguided. Maybe we’re making some bad assumptions. Maybe, just maybe, we’re asking the wrong questions. The interviewer has a lot of power over what to ask and when to ask it during open-ended interviews and sometimes even the most experienced interviewer can make poor decisions. This article will cover some key interviewing pitfalls to avoid when conducting a qualitative interview. It isn’t an exhaustive list, so if you have any that I’ve forgotten or anything here that you disagree with, I’d love to hear from you!

Leading questions

Leading questions might be the first thing that comes to mind when thinking about interviews gone wrong, and researchers fall prey to asking leading questions all the time. They can come in a few, devious forms.

Leaning in on an answer. This is what most people think of when they think about leading questions: phrasing the question in a way that assumes the answer and in doing so, influences the participant to respond in a certain way. An example might be, “Do you wear your motorcycle helmet to give you a sense of security?” A better alternative might be, “Why do you wear a motorcycle helmet?” Leading questions shouldn’t be confused with mirroring questions (or “parroting” or “echoing” questions), confirming what the participant just said for clarification or to prompt additional information. The mirroring version would be something like, “What you’re saying is that wearing your motorcycle helmet gives you a sense of security?” That’s perfectly fine as long as you’re not sneaking little assumptions into your confirmations.

Narrowing the possibilities. Another way interviewers can lead a participant is by listing, and thereby limiting, potential responses. Here the interviewer may have the best intentions and funnel their participant unwittingly. Sometimes the participant is struggling with answering, and it can seem helpful to follow up with some possible direction. An example might be, “How do you use your cell phone…Do you text? Surf the web? Play games?”

These follow-up questions are unfortunate. For one, it can be helpful to know their first, top-of-mind answer, indicating it might be their most frequent or most important use. For me, if someone asked how I use my phone, my first answer might be, “I look to see if there are any good Burger King coupons.” If someone attached the list of examples to their question, I might start with “texting” instead and never even make it to my fast food addiction.

If you need specific information about their texting, you might start with an open-ended question (“What do you use your phone for?”) and if they don’t mention it on the first go around, you can follow up with questions about their texting habits.

Social influence. Social desirability bias is also an issue in quantitative research, but it can be especially potent in face-to-face, qualitative research. We often share our own experiences during our interviews and usually this is a totally harmless way to make the relationship more equitable and to build rapport. The danger is that it can key them into how we feel about whatever we’re studying and potentially steer their answers. It isn’t so much about asking the wrong questions as it is about being careful of what personal opinions to share. Studying politics? Don’t go in wearing your MAGA hat or your BLM shirt. Studying snacks? Don’t go in with orange fingers and lips, asking questions like, “I love Cheetos a lot, what do you think about them?”

Clustered questions

Another interview sin is bombarding participants with multiple questions at once. It can be a little overwhelming for the recipient of the onslaught, more like a college exam than a friendly conversation. Often the participant will answer just one of the questions, and maybe the interviewer will remember to return to the others, but maybe not.

Baffling questions

Ethnographers may be more prone to baffling questions than some other qualitative researchers. We are more likely to be looking at things like culture, social structures, and semiotics, and if we’re not careful, our questions can be overly abstract. Sometimes, we make the error of asking participants the same sort of “how the world works” questions that we’re asking ourselves in our analysis.

Questions might baffle participants when we use terms or concepts that only social scientists or marketers would understand. We can get so used to our jargon that we forget that it is jargon. Our questions also might be so removed from our participants’ experiences that they might seem absurd or even laughable to them. Market researchers are always asking people to think deeply about products and behaviors they normally don’t give much thought. “What kind of unmet needs do you have with your sour cream?” “How does taking your pills in the morning impact your sense of self?” Whatever package the baffling question comes wrapped in, the confused look of participants is always the same. Time to rephrase or move on.

 Giant leap questions

Sometimes interviewers will change subjects so abruptly that it can throw off the pace and rapport of the interview. A segue can usually save the day but making a major shift in the interview’s direction can signal that you weren’t all that interested in what the participant just said. There will be times when leaps are necessary, e.g., if you have a participant who is bound and determined to talk about everything except what you’re there to talk about. There will also be times where a conversation line will be exhausted, and a new direction will seem natural. For the most part, however, questions should move the conversation along in a steady stride and not giant leaps.

Poorly-timed questions

Sometimes it isn’t about what you ask but about when you ask it. For one, interrupting participants is obviously in bad form, but interviewers might not even realize they’re doing it if they’re used to having to talk over their chatty friends and family. Not only must we wait our turn to talk, but we have to learn to embrace silence. It is a common technique to wait just a bit longer than seems natural to ask your next question with the idea that the discomfort from the silence will prompt the participant to elaborate further. It works, but at the very least, interviewers should give their participants plenty of breathing room to say what they please. 

Another example of bad timing is going too deep or too personal with participants too fast. You have to build a little trust before you start asking about sensitive subjects. This includes asking for a home tour too early. Bedrooms and bathrooms can be just as private and personal as talking about your embarrassing health issues or your deepest feelings.

“Normally” questions

It is better to ask someone what they actually do than what they normally do. For example, instead of asking someone, “What do you usually eat for dinner?” You might ask them, “What did you have for dinner last night?” Or, “What’s on the menu for next week?” Afterwards, you might follow up with a question on how that compares to their typical dinners, but this approach promotes detailed answers and specific examples that can give color to your understanding. Or, better yet, skip the questions altogether and ask to join them for dinner or have them record their meals on a video diary!

Questions with one-word answers

Finally, when you’re doing open-ended interviewing, it is good practice to avoid questions that can be answered with one-word replies (unless you’re looking for very specific information). Instead, questions should be framed in a way that prompts as much elaboration as possible. For example, a better option to, “Did cost factor into your purchase?” might be, “Tell me how cost factored into your purchase?” Alternatively, you can follow up your more close-ended questions with something open-ended like, “Tell me more about that.” 

In fact, asking someone to “tell me more” is the ultimate open-ended prompt. It isn’t really a question at all but a simple invitation to talk and share. This is the sweet spot of the ethnographic interview, where the interviewer simply provides the fuel for the participant to share themselves and their stories. It’s unrealistic to avoid questions that seek specific information entirely, but the participant exploring their relationship with the research subject while the ethnographer is there only as a delicate guide is at the heart of all good ethnographic interviews. As long as you are conducting your research in that spirit, all of your questions, more likely than not, will be good questions.

Categories: Ethnography, Process