Todd Hollander Market Research

In online surveys, open-ended questions are those in which the respondents type their responses in a text box rather than selecting from a set of predefined responses such as radio buttons, checkboxes, or drop-down lists. When used properly, open-ended questions can provide several benefits. However, when used improperly, they can be the kiss of death for your survey.

Pros
Open-ended questions are useful when the range of responses is not tightly defined and/or detailed responses are desired. Advantages include a wider variety and greater depth of responses, and a higher likelihood of receiving unexpected and insightful information. They can also be used to capture any important ideas or opinions that you may not have thought to ask about.

Cons
Open-ended questions have several disadvantages. First, the responses must be read individually because there is no effective way to automatically tabulate or perform statistical analyses on them. This is more expensive and time consuming than the analysis of closed-ended questions and may not be practical for low-budget or time-sensitive research.

These responses are also open to the interpretation of the reader because different people are likely to interpret the meaning of a response in different ways. This potential problem can be avoided by using a single analyst, but a large number of responses can make this impractical.

The most significant drawback of using open-ended questions is that it may significantly increase the number of incompleted interviews (when respondents exit the survey before responding to all of the questions). This is because open-ended questions require more thought, time, and effort from the respondent.

Inexperienced researchers often make the mistake of including too many open-ended questions in a survey. It is important to understand that in most cases respondents are less passionate about the subject matter than the sponsor of the research. If your survey exceeds their tolerance of time and effort, they are likely to abandon the interview.

So how many open-ended questions should be included in a survey?
This is an important question. Unfortunately, there is no simple response. The number of open-ends your respondents will tolerate generally depends on:

• The total length of the interview
• The incentive for completing the survey
• How much they care about the subject matter
• The extent to which they believe their responses will be read and considered

The bottom line: Open-ended questions can be highly beneficial but should be used sparingly. Using too many can significantly decrease the response rate for your survey.

See also: How and when to use text boxes in online surveys

This entry was posted on Sunday, February 24th, 2008 at 8:07 pm and is filed under Questionnaire Design. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.