Fatal flaw in online survey design: Leading questions
Email This Post
Posted by: Todd Hollander in Questionnaire Design
A leading question (also known as a “loaded question”) is one that suggests an answer by the way in which the question or response options are worded. Here’s an example of leading the respondent by the way the question is worded:
How high would you rate this product?
Asking the question in this way leads the respondent to offer a high rating. That is, it sets the expectation that a high rating is the “correct” response. To ask this question in a neutral (i.e., non-leading) way, the word “high” should be eliminated to render the question: “How would you rate this product?” Often a survey question is loaded by an unbalanced set of response options. Take this question for example:
How would you rate this product
Excellent
Great Good Fair Poor
Notice that the question itself is not leading but the response options are. This is because four of the five possible responses are positive and only one (”Poor”) is negative. A closed-ended question should include response options that not only cover the whole range of responses, but that are also equally distributed throughout the range. That is, all responses should be equally likely. Thus, a more appropriate scale would be:
Very Good Good Neutral Bad Very Bad
This balanced set of response options does not imply an expectation of either a positive or negative response. Nor does it stack the odds that responses will cluster at one end of the spectrum.
The bottom line: To avoid the common mistake of asking leading questions, make sure that none of your survey questions suggest an answer by the way in which the question is worded, and that each question provides response options that not only cover the whole range of responses, but are also equally distributed throughout the range.
Entries (RSS)