Todd Hollander Market Research

An online survey can be formatted so that all the questions appear on a single page or across several pages. Deciding which format to use for your survey can have a significant effect on both the response rate and the quality of information you receive.

(more…)

When planning a quantitative market research study, one of the first decisions that must be made is which method of data collection to use: online, telephone, in-person, postal mail, email, or other. Until the mid 1990’s, telephone interviewing was the norm for market researchers. Today, online surveys have become the preferred method for most research projects.

Using the Internet as a medium for data collection began around 1995. In its infancy, online surveying was considered inferior to offline methods such as telephone interviewing because of concerns about:

• Representativeness. Only a portion of the population had access to the internet.
• Reaching the respondent . There were no directories or complete lists of Internet users from which samples could be drawn.
• Unsupervised respondents. There was no way to verify who was really at the other end of the connection or how seriously the respondent took the survey.
• Technical limitations of the medium. Slow connections, conflicting security settings, lack of plug-ins for advanced features, differing hardware and software configurations, and bottlenecks caused by too many people trying to access a survey at the same time.
• Loss of direct contact. Interviewers could no longer use and benefit from direct verbal communication with the respondent.
• Privacy concerns. Some respondents were skeptical about the confidentiality of their responses or how the information would be used.
• “Professional” respondents. The integrity of the results could be compromised by respondents who were motivated primarily by incentives.

In the last decade, online research has evolved and expanded to solve each of these problems. As a result, online surveys have replaced telephone interviewing as the “gold standard” of market research because they offer:

• Lower cost
• Faster turnaround
• More convenience and less intrusion for respondents
• Ability to gain insight from hard-to-reach people and geographically dispersed populations
• A more flexible medium, including the increasingly wide range of stimuli that can be used
• More thorough responses
• More truthful/revealing responses

The bottom line: Online surveys offer a number of advantages over telephone interviews and other offline methods. Although no single data collection method is best for all situations, online surveys have become the “gold standard” of marketing research.

E-mail invitations are the preferred method for requesting participation in online surveys. Here are a few tips for making sure your email invitation is well received and produces the results you want.

Email invitations for online surveys often include a personalized greeting. It is standard practice to include a hyperlink to the survey so respondents simply click on the link to launch the survey.

As a rule, the survey invitation should communicate:

1. That the recipient is being invited to take part in a marketing research study
2. The purpose of the research
3. Who is conducting the research
4. How to complete the survey
5. How long the survey will take
6. What the the incentive is (if any)
7. How the privacy of survey respondents is protected
8. That the recipient can opt out of further contact with the sender

I am often asked how much information about the survey should be revealed in the invitation. My advice is to include just enough information about the survey to garner an acceptable level of participation but not so much that you tip off potential respondents on either the screening requirements for completing the survey or the specific questions that will be asked. Here’s why: If the invitation contains too little information about the the survey and the incentive for completing it, the response rate may be too low; conversely, if too much information is revealed, some unqualified respondents could modify their responses to screening questions in order to ensure that they are allowed to participate.

Sample Invitation

From: jsmith@acmeresearch.com

Subject: Request for Help with Toothbrush Research

Message:

Dear Mrs. Jones,

You are invited to help improve a leading brand of toothbrush by taking part in a short survey. Please click www.acmeresearch.com/toothbrush to launch the survey.

As a token of our gratitude for taking part in this survey, you will be eligible to enter your name into a prize drawing for one of five $100 prepaid gas cards.

This survey is genuine marketing research. It should take no more than 10 minutes to complete and is being conducted in accordance with the CASRO guidelines. This that means your identity will not be revealed to any other company, and you will receive NO marketing as a result of taking part in this survey. If you want to know more about CASRO or Acme Research, click on one of the appropriate links at the bottom of this message.

To start the survey, please click this link: testsurvey.

Thanks and best regards,

John Smith
President
ACME Corporation
Address
City, ST, Zip
Toll-free phone: xxx
E-mail: jsmith@acmeresearch.com

To find out more about ACME Research, click here www.acmeresearch.com.

To read the CASRO guidelines, click here www.casro.org/codeofstandards.cfm.

To read the terms and conditions of the prize drawing associated with this survey, click here www.acmeresearch.com/project32/tc.

You were selected for this study from a list supplied by XYZ corporation. If you would like to find out more about XYZ or request that you be removed from their list, click www.XYZ.com/Privacy.htm

The bottom line: A well-constructed email invitation will not only increase the response rate for your online survey, but will also improve the the quality of data you receive from respondents.

Appropriate formatting is a vital component of effective online surveys. The right formatting will engage respondents, make it easier for them to navigate through the survey, and increase the survey response rate. Here are a few tips for how and when to use grid responses.

(more…)

Appropriate formatting is a vital component of effective online surveys. The right formatting will engage respondents, make it easier for them to navigate through the survey, and increase the survey response rate. Here are a few tips for how and when to use text boxes.

Text Box

Text boxes are blank spaces in which users can enter free-form answers. Text boxes are generally used when a) the researcher wants to capture respondents’ opinions in their own words, and/or b) there are more response options than can be accommodated by a drop-down list. Recommendations:

• Use text boxes sparingly. As a rule, use a text box only if you cannot use a closed-ended question format (radio buttons, checkboxes, drop-down lists, or grids).
• Because it takes respondents longer to complete text boxes than radio buttons or checkboxes, the more text boxes you include, the more incomplete surveys you are likely to receive because some respondents will consider the survey too time-consuming and taxing.
• Providing clear, concise instructions tends to produce the best results.
• Placement of text boxes is important. As a rule, you should avoid using them too early in the survey because most respondents prefer to start with easy single or multiple-choice questions.
• Respondents often fill the space provided, so it is important to allow sufficient space. If you want shorter answers, limit the space provided. If you want longer responses, use larger boxes to accommodate longer responses.
• Making text box responses optional increases the likelihood that respondents will not complete them, but requiring responses in text boxes that should be optional (such as "Please use this space for any additional comments") increases the likelihood that frustrated respondents will not complete the survey.

Appropriate formatting is a vital component of effective online surveys. The right formatting will engage respondents, make it easier for them to navigate through the survey, and increase the survey response rate. Here are a few tips for how and when to use drop-down lists.

Drop-down list

Drop-down lists typically appear in the survey as a box with instructions to "select one." This type of response format requires the user to click on the box, then locate and click on their answer. Drop-down lists are most appropriate when a) the number of response options is too large to format with radio buttons, and/or b) it is faster or easier for the respondent to select a response from a drop-down list rather than a list of radio buttons. Recommendations:

• Because all of the possible responses are not visible on the initial screen, drop-down options should be used for fields with which respondents are already familiar (such as year of birth or state of residence).
• When many possible responses are included in the drop-down list, it is important to organize response options in a logical way (e.g., alphabetically or numerically).
• Use drop-down lists when appropriate as research has shown that they do not increase the likelihood of respondents giving “don’t know” responses or leaving items blank. ^*

——————————————
* Couper, Tourangeau, Conrad and Crawford. “What They See is What We Get.” Social Science Computer Review. http://ssc.sagepub.com/cgi/reprint/22/1/111.pdf (2004).

Appropriate formatting is a vital component of effective online surveys. The right formatting will engage respondents, make it easier for them to navigate through the survey, and increase the survey response rate. Here are a few tips for how and when to use checkboxes.

Checkboxes

Checkboxes are used for multiple-choice questions. These response categories usually are not mutually exclusive and are used when multiple answers exist for a single question, as in the example on the left. Note that checkboxes are used when there are lists of options and the user may select any number of choices, including none, one, several, or all. In other words, each checkbox is independent of all other checkboxes in the list, so checking one box doesn’t uncheck the others.

Appropriate formatting is a vital component of effective online surveys. The right formatting will engage respondents, make it easier for them to navigate through the survey, and increase the survey response rate. Here are a few tips for how and when to use radio buttons.

Radio Buttons

In web survey design, radio buttons are represented as small round circles. Radio buttons should be used when the respondent is to select only one answer from a predefined set of options, as in the demographic question on the left: As a rule, radio button responses should be mutually exclusive. A few interesting findings:

• A 2002 experiment in Belgium demonstrated one advantage of using radio buttons. In this experiment, two groups were given the same survey: one with radio buttons and the other with drop-down lists. Participants using radio buttons were more likely to complete the survey (88.37 percent) than those using drop-downs (84.07 percent)1 .
• The use of radio buttons that offer the user noncommittal answers such as “don’t know” did not increase the likelihood of such non-substantial answers being selected1 .

In future postings, we’ll explore how and when to use the alternatives to radio buttons, including:

• Checkboxes
• Drop-down lists
• Text boxes
• Grids

__________________________
1Heerwegh and Loosveldt. “An Evaluation of the Effect of Response Format on Data Quality in Web Surveys.” www.icis.dk/ICIS_papers/A2_3_2.pdf (2002) and Social Science Computer Review, Vol. 20, No. 4, 471-484 (2002)

Between my clients at Todd Hollander Market Research and my students in the Priniciples of Marketing Research course, I am constantly asked “What is the best scale to use for customer satisfaction surveys?” The short answer is “There isn’t one.”

The question about which scale is best usually refers to noncomparative scales. In noncomparative scaling (the most common scaling technique for market research), each object is rated on its own scale independently of the others in the stimulus set. For example, respondents may be asked to rate a product attribute on a 1 to 7 scale where a rating of “1″ = “Poor” and a rating of “7″ = “Excellent.” This is the type of rating that begs the question about which scale is best. The question is usually posed along the lines of “Is a 7-point scale better than a 5-point scale?” or “Is a 10-point scale better than a 7-point scale?”

In reality, there simply isn’t one rating scale (5-point, 7-point, etc.) that is preferred over all others for satisfaction surveys (or most other types of surveys). Why not? Because the ideal scale depends on a combination of:

• The question being asked
• How knowledgeable respondents are about the subject of the question
• The number of completed surveys anticipated
• How the results will be analyzed
• How the results will be used in decision-making

Using these factors to construct effective rating scales takes considerable skill and experience. It is also important to know how to transition respondents from one rating scale (e.g. 5-point) to another (e.g. 10-point) without causing confusion. When inappropriate scales are used, or when multiple scales are used in a single survey, respondents may perceive the survey as illogical and/or unprofessional.

The bottom line: There is no “best” rating scale for satisfaction surveys. Knowing which scales to use and how to use them effectively takes a considerable amount of experience.