Todd Hollander Market Research

When planning a quantitative market research study, one of the first decisions that must be made is which method of data collection to use: online, telephone, in-person, postal mail, email, or other. Until the mid 1990’s, telephone interviewing was the norm for market researchers. Today, online surveys have become the preferred method for most research projects.

Using the Internet as a medium for data collection began around 1995. In its infancy, online surveying was considered inferior to offline methods such as telephone interviewing because of concerns about:

• Representativeness. Only a portion of the population had access to the internet.
• Reaching the respondent . There were no directories or complete lists of Internet users from which samples could be drawn.
• Unsupervised respondents. There was no way to verify who was really at the other end of the connection or how seriously the respondent took the survey.
• Technical limitations of the medium. Slow connections, conflicting security settings, lack of plug-ins for advanced features, differing hardware and software configurations, and bottlenecks caused by too many people trying to access a survey at the same time.
• Loss of direct contact. Interviewers could no longer use and benefit from direct verbal communication with the respondent.
• Privacy concerns. Some respondents were skeptical about the confidentiality of their responses or how the information would be used.
• “Professional” respondents. The integrity of the results could be compromised by respondents who were motivated primarily by incentives.

In the last decade, online research has evolved and expanded to solve each of these problems. As a result, online surveys have replaced telephone interviewing as the “gold standard” of market research because they offer:

• Lower cost
• Faster turnaround
• More convenience and less intrusion for respondents
• Ability to gain insight from hard-to-reach people and geographically dispersed populations
• A more flexible medium, including the increasingly wide range of stimuli that can be used
• More thorough responses
• More truthful/revealing responses

The bottom line: Online surveys offer a number of advantages over telephone interviews and other offline methods. Although no single data collection method is best for all situations, online surveys have become the “gold standard” of marketing research.

While no amount of research can ever completely eliminate the risk of deploying a marketing strategy, effective marketing research should significantly reduce uncertainty and financial risk. Here are a few tips for developing a marketing research project that pays off.

(more…)

According to a study by the Pew Research Center, 12.8 percent of U.S. households cannot be reached by a telephone survey because they have only a cell phone and no landline telephone. If these people were just like those with landlines, this would not create a problem for researchers. But cell phone-only adults are very different.

Cell Phone Only

A growing number of Americans rely solely on a cell phone for their telephone service, and many more are considering giving up their landline phones. This trend presents a challenge to market research and public opinion polling conducted by telephone, which typically relies on a sample of the population of landline subscribers.

The National Health Interview Survey, conducted by the Pew Research Center in conjunction with the Associated Press and AOL, found cell phone-only adults to be younger, more heavily African-American or Hispanic, less likely to be married, and less likely to own a home than adults with landline telephones. These demographic characteristics were correlated with a wide range of social and political behaviors.

In early 2003, just 3.2 percent of households were cell phone-only. In the fall of 2004, the National Election Pool’s exit poll found that 7.1 percent of those who voted on Election Day had only a cell phone. At the annual meeting of the American Association for Public Opinion Research, a government researcher predicted that the size of the cell phone-only group could approach 25 percent by the end of 2008 if the current rate of increase is sustained.

Landline Only

According to data from the National Center for Heath Statistics, 37% of Americans have only a landline. Like the cell phone-only population, Americans who use only a landline only are demographically distinctive. A much higher percentage (41%) are ages 65 and older, compared with the general public (16%). The landline-only group also includes a greater proportion of whites (82%) than the general public (73%).

About a quarter of landline users (23%) say they are very (8%) or somewhat likely (15%) to stop using their landline and switch to using only a cell phone. A much higher percentage of young people (40% of those under age 30 vs. 19% of those 30 and older) say they are likely to abandon their landlines.

The Bottom Line

Political polls can be weighted to offset the effect of the cell phone-only population. However, because of the rising number of Americans who no longer use landline phones, there is growing concern about how long the landline telephone survey will remain a viable data collection tool for traditional market research.

E-mail invitations are the preferred method for requesting participation in online surveys. Here are a few tips for making sure your email invitation is well received and produces the results you want.

Email invitations for online surveys often include a personalized greeting. It is standard practice to include a hyperlink to the survey so respondents simply click on the link to launch the survey.

As a rule, the survey invitation should communicate:

1. That the recipient is being invited to take part in a marketing research study
2. The purpose of the research
3. Who is conducting the research
4. How to complete the survey
5. How long the survey will take
6. What the the incentive is (if any)
7. How the privacy of survey respondents is protected
8. That the recipient can opt out of further contact with the sender

I am often asked how much information about the survey should be revealed in the invitation. My advice is to include just enough information about the survey to garner an acceptable level of participation but not so much that you tip off potential respondents on either the screening requirements for completing the survey or the specific questions that will be asked. Here’s why: If the invitation contains too little information about the the survey and the incentive for completing it, the response rate may be too low; conversely, if too much information is revealed, some unqualified respondents could modify their responses to screening questions in order to ensure that they are allowed to participate.

Sample Invitation

From: jsmith@acmeresearch.com

Subject: Request for Help with Toothbrush Research

Message:

Dear Mrs. Jones,

You are invited to help improve a leading brand of toothbrush by taking part in a short survey. Please click www.acmeresearch.com/toothbrush to launch the survey.

As a token of our gratitude for taking part in this survey, you will be eligible to enter your name into a prize drawing for one of five $100 prepaid gas cards.

This survey is genuine marketing research. It should take no more than 10 minutes to complete and is being conducted in accordance with the CASRO guidelines. This that means your identity will not be revealed to any other company, and you will receive NO marketing as a result of taking part in this survey. If you want to know more about CASRO or Acme Research, click on one of the appropriate links at the bottom of this message.

To start the survey, please click this link: testsurvey.

Thanks and best regards,

John Smith
President
ACME Corporation
Address
City, ST, Zip
Toll-free phone: xxx
E-mail: jsmith@acmeresearch.com

To find out more about ACME Research, click here www.acmeresearch.com.

To read the CASRO guidelines, click here www.casro.org/codeofstandards.cfm.

To read the terms and conditions of the prize drawing associated with this survey, click here www.acmeresearch.com/project32/tc.

You were selected for this study from a list supplied by XYZ corporation. If you would like to find out more about XYZ or request that you be removed from their list, click www.XYZ.com/Privacy.htm

The bottom line: A well-constructed email invitation will not only increase the response rate for your online survey, but will also improve the the quality of data you receive from respondents.

In order to achieve a high level of confidence in the results of online surveys, it is vitally important to maximize the percentage of potential respondents who actually complete the survey.

Here are 10 tips that will help you increase survey response rates.

(more…)

Appropriate formatting is a vital component of effective online surveys. The right formatting will engage respondents, make it easier for them to navigate through the survey, and increase the survey response rate. Here are a few tips for how and when to use grid responses.

(more…)

Appropriate formatting is a vital component of effective online surveys. The right formatting will engage respondents, make it easier for them to navigate through the survey, and increase the survey response rate. Here are a few tips for how and when to use text boxes.

Text Box

Text boxes are blank spaces in which users can enter free-form answers. Text boxes are generally used when a) the researcher wants to capture respondents’ opinions in their own words, and/or b) there are more response options than can be accommodated by a drop-down list. Recommendations:

• Use text boxes sparingly. As a rule, use a text box only if you cannot use a closed-ended question format (radio buttons, checkboxes, drop-down lists, or grids).
• Because it takes respondents longer to complete text boxes than radio buttons or checkboxes, the more text boxes you include, the more incomplete surveys you are likely to receive because some respondents will consider the survey too time-consuming and taxing.
• Providing clear, concise instructions tends to produce the best results.
• Placement of text boxes is important. As a rule, you should avoid using them too early in the survey because most respondents prefer to start with easy single or multiple-choice questions.
• Respondents often fill the space provided, so it is important to allow sufficient space. If you want shorter answers, limit the space provided. If you want longer responses, use larger boxes to accommodate longer responses.
• Making text box responses optional increases the likelihood that respondents will not complete them, but requiring responses in text boxes that should be optional (such as "Please use this space for any additional comments") increases the likelihood that frustrated respondents will not complete the survey.

Appropriate formatting is a vital component of effective online surveys. The right formatting will engage respondents, make it easier for them to navigate through the survey, and increase the survey response rate. Here are a few tips for how and when to use drop-down lists.

Drop-down list

Drop-down lists typically appear in the survey as a box with instructions to "select one." This type of response format requires the user to click on the box, then locate and click on their answer. Drop-down lists are most appropriate when a) the number of response options is too large to format with radio buttons, and/or b) it is faster or easier for the respondent to select a response from a drop-down list rather than a list of radio buttons. Recommendations:

• Because all of the possible responses are not visible on the initial screen, drop-down options should be used for fields with which respondents are already familiar (such as year of birth or state of residence).
• When many possible responses are included in the drop-down list, it is important to organize response options in a logical way (e.g., alphabetically or numerically).
• Use drop-down lists when appropriate as research has shown that they do not increase the likelihood of respondents giving “don’t know” responses or leaving items blank. ^*

——————————————
* Couper, Tourangeau, Conrad and Crawford. “What They See is What We Get.” Social Science Computer Review. http://ssc.sagepub.com/cgi/reprint/22/1/111.pdf (2004).

Appropriate formatting is a vital component of effective online surveys. The right formatting will engage respondents, make it easier for them to navigate through the survey, and increase the survey response rate. Here are a few tips for how and when to use checkboxes.

Checkboxes

Checkboxes are used for multiple-choice questions. These response categories usually are not mutually exclusive and are used when multiple answers exist for a single question, as in the example on the left. Note that checkboxes are used when there are lists of options and the user may select any number of choices, including none, one, several, or all. In other words, each checkbox is independent of all other checkboxes in the list, so checking one box doesn’t uncheck the others.

Appropriate formatting is a vital component of effective online surveys. The right formatting will engage respondents, make it easier for them to navigate through the survey, and increase the survey response rate. Here are a few tips for how and when to use radio buttons.

Radio Buttons

In web survey design, radio buttons are represented as small round circles. Radio buttons should be used when the respondent is to select only one answer from a predefined set of options, as in the demographic question on the left: As a rule, radio button responses should be mutually exclusive. A few interesting findings:

• A 2002 experiment in Belgium demonstrated one advantage of using radio buttons. In this experiment, two groups were given the same survey: one with radio buttons and the other with drop-down lists. Participants using radio buttons were more likely to complete the survey (88.37 percent) than those using drop-downs (84.07 percent)1 .
• The use of radio buttons that offer the user noncommittal answers such as “don’t know” did not increase the likelihood of such non-substantial answers being selected1 .

In future postings, we’ll explore how and when to use the alternatives to radio buttons, including:

• Checkboxes
• Drop-down lists
• Text boxes
• Grids

__________________________
1Heerwegh and Loosveldt. “An Evaluation of the Effect of Response Format on Data Quality in Web Surveys.” www.icis.dk/ICIS_papers/A2_3_2.pdf (2002) and Social Science Computer Review, Vol. 20, No. 4, 471-484 (2002)