Clichés and aphorisms can be both inspirational and educational. Sometimes they are practical for everyday living. Take, for example, the sayings: “There are no stupid questions” or “The only stupid question is the one you don’t ask.”
In market research, however, clichés and aphorism are like sailing on the Titanic. Rest assured this is the case for survey questions. You can ask stupid questions and there are questions you shouldn’t ask. One of the foundations of market research is based on the collection of accurate data, and a poorly-written questionnaire will likely deliver substandard information. Therefore, expect some very accurate but pointed questions to you by leadership when campaigns or studies miss the mark.
LEARN ABOUT: Open-Ended Questions
To collect the right data, start by asking the right questions, by avoiding these stupid question types:
Biased/Leading Questions
For this research oversight, we can bring the ole law cliché: “Where were you on the night you murdered your spouse?”
For a clear definition of biased/leading questions, Research Access offers:
“A leading question suggests the answer the survey author is looking for and often unintentionally reflects the author’s bias. As a result, the answers to such questions overstate actual support for the item being researched.”
A perfect example today could be:
- Are you planning on watching Black Panther, a film that is both culturally impactful and record-breaking?
Yes, the movie is outstanding in many ways and deserves meaningful conversations among friends in a coffee shop. How about this question instead:
- Are you planning on watching Black Panther?
Keep it simple, bringing in one last cliché.
Double-barreled Questions
In a mobile device era, it’s a sound approach to make questionnaires shorter, with research showing that for most projects, market researchers can now expect up to 40% response on the small screens of smartphones. It’s just sound to keep it short, period, considering these short-attention-span, media-competitive times.
Still, what often happens in surveys is that researchers will stuff content to compress length. And what we get is two questions masked as one, which is the double-barreled question.
For instance, one might ask:
- Where you able to get your soy-skim-tepid-decaf-organic latte on time and did it taste great?
When it’s better to break the line into two questions:
- Where you able to get your soy-skim-tepid-decaf-organic latte?
- Did your soy-skim-tepid-decaf-organic latte taste great?
And then maybe give baristas around the world a cyber hug.
Straight-To-Third Date Questions
Generally speaking, researchers should start surveys with general questions. This allows respondents to get comfortable with the research, open up a bit.
Therefore, personal questions like “What is your income?” or socially-sensitive questions should come towards the end of a questionnaire. Think of this approach as being in a relationship with respondents — with marriage something to engage in down the line.
Yes/No Questions
We mentioned the benefits of simplicity, and this is not the right context to apply this! Yes/no questions are difficult to answer and can negate context. This also includes either/or questions.
As marketer Jeff Haden said in an Inc. article: “Either/or questions, just like leading questions, assume some answer.”
On a more scientific level, a researcher states on avoiding yes/no questions:
“Decades of research in survey methodology and psychology have shown that people generally tend to avoid saying “no.” In the survey context, this is called acquiescence response bias and it is a serious threat to data quality.”
Let’s face it: people are beings full of experiences and varying degrees of emotion — so offering scales is a better alternative. Give your respondents a range to give your research precise data.
Double Negative/Confusing Questions
This questionnaire lapse is obvious, but sometimes researchers (just like writers) get so close to a project they overlook the obvious.
An example would be:
- How do you feel about the following statement? We should not reduce government Skynet research.
Instead it should be:
- How do you feel about the following statement? We should reduce government Skynet research.
Answers that don’t match questions, indirect questions (or ones without an open-ended/other option), or mutually inclusive responses (like a line for “spouse” and one for “relative”) will dilute survey data to the point research goes, “Hasta la vista, baby.”
Beyond continued editing, an extra set of eyes is always a sensible idea when crafting a questionnaire. And nothing wrong with reading out loud if it doesn’t annoy your cubicle-neighbor!
Mandatory Questions
Required questions should be avoided like a North Korean spa. Sure, every question is essential for market research. The thing is, forcing the issue with the dreaded “Please select an answer” button can likely cause respondents to go on a random-answering spree or even bail on the survey altogether.
As one market researcher put very well:
“Respondents are giving their time and effort to provide information. You are being selfish and fooling yourself by requiring them to respond to all questions.”
At the very least, offer an open-ended option. Proceed at your own peril.
LEARN ABOUT: This or that questions
Adding the “Smart” to Questions
Even after avoiding these question blunders, here are some parting tips from our research that will enhance your questionnaire for more quality data:
- Keep your questions as short and concise as possible
- Avoid complex matrices
- Include open-ended questions
- Keep surveys at 20 questions or less
- Offer incentives if possible (which typically result in 10-15% higher response rates)
- Start with the essential questions (if they aren’t too personal!)
With all of this in mind, you’ll have better results from your research, and more time to get back to life and its clichés.
