Todd Hollander Market Research

Appropriate formatting is a vital component of effective online surveys. The right formatting will engage respondents, make it easier for them to navigate through the survey, and increase the survey response rate. Here are a few tips for how and when to use grid responses.

(more…)

Appropriate formatting is a vital component of effective online surveys. The right formatting will engage respondents, make it easier for them to navigate through the survey, and increase the survey response rate. Here are a few tips for how and when to use text boxes.

Text Box

Text boxes are blank spaces in which users can enter free-form answers. Text boxes are generally used when a) the researcher wants to capture respondents’ opinions in their own words, and/or b) there are more response options than can be accommodated by a drop-down list. Recommendations:

• Use text boxes sparingly. As a rule, use a text box only if you cannot use a closed-ended question format (radio buttons, checkboxes, drop-down lists, or grids).
• Because it takes respondents longer to complete text boxes than radio buttons or checkboxes, the more text boxes you include, the more incomplete surveys you are likely to receive because some respondents will consider the survey too time-consuming and taxing.
• Providing clear, concise instructions tends to produce the best results.
• Placement of text boxes is important. As a rule, you should avoid using them too early in the survey because most respondents prefer to start with easy single or multiple-choice questions.
• Respondents often fill the space provided, so it is important to allow sufficient space. If you want shorter answers, limit the space provided. If you want longer responses, use larger boxes to accommodate longer responses.
• Making text box responses optional increases the likelihood that respondents will not complete them, but requiring responses in text boxes that should be optional (such as "Please use this space for any additional comments") increases the likelihood that frustrated respondents will not complete the survey.

Appropriate formatting is a vital component of effective online surveys. The right formatting will engage respondents, make it easier for them to navigate through the survey, and increase the survey response rate. Here are a few tips for how and when to use drop-down lists.

Drop-down list

Drop-down lists typically appear in the survey as a box with instructions to "select one." This type of response format requires the user to click on the box, then locate and click on their answer. Drop-down lists are most appropriate when a) the number of response options is too large to format with radio buttons, and/or b) it is faster or easier for the respondent to select a response from a drop-down list rather than a list of radio buttons. Recommendations:

• Because all of the possible responses are not visible on the initial screen, drop-down options should be used for fields with which respondents are already familiar (such as year of birth or state of residence).
• When many possible responses are included in the drop-down list, it is important to organize response options in a logical way (e.g., alphabetically or numerically).
• Use drop-down lists when appropriate as research has shown that they do not increase the likelihood of respondents giving “don’t know” responses or leaving items blank. ^*

——————————————
* Couper, Tourangeau, Conrad and Crawford. “What They See is What We Get.” Social Science Computer Review. http://ssc.sagepub.com/cgi/reprint/22/1/111.pdf (2004).

Appropriate formatting is a vital component of effective online surveys. The right formatting will engage respondents, make it easier for them to navigate through the survey, and increase the survey response rate. Here are a few tips for how and when to use checkboxes.

Checkboxes

Checkboxes are used for multiple-choice questions. These response categories usually are not mutually exclusive and are used when multiple answers exist for a single question, as in the example on the left. Note that checkboxes are used when there are lists of options and the user may select any number of choices, including none, one, several, or all. In other words, each checkbox is independent of all other checkboxes in the list, so checking one box doesn’t uncheck the others.

Appropriate formatting is a vital component of effective online surveys. The right formatting will engage respondents, make it easier for them to navigate through the survey, and increase the survey response rate. Here are a few tips for how and when to use radio buttons.

Radio Buttons

In web survey design, radio buttons are represented as small round circles. Radio buttons should be used when the respondent is to select only one answer from a predefined set of options, as in the demographic question on the left: As a rule, radio button responses should be mutually exclusive. A few interesting findings:

• A 2002 experiment in Belgium demonstrated one advantage of using radio buttons. In this experiment, two groups were given the same survey: one with radio buttons and the other with drop-down lists. Participants using radio buttons were more likely to complete the survey (88.37 percent) than those using drop-downs (84.07 percent)1 .
• The use of radio buttons that offer the user noncommittal answers such as “don’t know” did not increase the likelihood of such non-substantial answers being selected1 .

In future postings, we’ll explore how and when to use the alternatives to radio buttons, including:

• Checkboxes
• Drop-down lists
• Text boxes
• Grids

__________________________
1Heerwegh and Loosveldt. “An Evaluation of the Effect of Response Format on Data Quality in Web Surveys.” www.icis.dk/ICIS_papers/A2_3_2.pdf (2002) and Social Science Computer Review, Vol. 20, No. 4, 471-484 (2002)

Between my clients at Todd Hollander Market Research and my students in the Priniciples of Marketing Research course, I am constantly asked “What is the best scale to use for customer satisfaction surveys?” The short answer is “There isn’t one.”

The question about which scale is best usually refers to noncomparative scales. In noncomparative scaling (the most common scaling technique for market research), each object is rated on its own scale independently of the others in the stimulus set. For example, respondents may be asked to rate a product attribute on a 1 to 7 scale where a rating of “1″ = “Poor” and a rating of “7″ = “Excellent.” This is the type of rating that begs the question about which scale is best. The question is usually posed along the lines of “Is a 7-point scale better than a 5-point scale?” or “Is a 10-point scale better than a 7-point scale?”

In reality, there simply isn’t one rating scale (5-point, 7-point, etc.) that is preferred over all others for satisfaction surveys (or most other types of surveys). Why not? Because the ideal scale depends on a combination of:

• The question being asked
• How knowledgeable respondents are about the subject of the question
• The number of completed surveys anticipated
• How the results will be analyzed
• How the results will be used in decision-making

Using these factors to construct effective rating scales takes considerable skill and experience. It is also important to know how to transition respondents from one rating scale (e.g. 5-point) to another (e.g. 10-point) without causing confusion. When inappropriate scales are used, or when multiple scales are used in a single survey, respondents may perceive the survey as illogical and/or unprofessional.

The bottom line: There is no “best” rating scale for satisfaction surveys. Knowing which scales to use and how to use them effectively takes a considerable amount of experience.