Todd Hollander Market Research

As an incentive for online survey participation, some companies offer to share the survey results with respondents. Here are the pros and cons for this type of incentive.

Pros
Offering to share some or all of the survey results with respondents can sometimes be effective with business-to-business research. For employees of certain companies, incentives such as guaranteed rewards and prize drawings are considered conflicts of interest and are therefore not permitted. Thus, the ability to share in the insight gleaned from the survey can be an extra incentive.

Cons
For consumer research, offering to share the results with respondents is rarely an effective incentive because most consumers prefer guaranteed rewards or prize drawings.

For business-to-business research, offering to share the results can increase the likelihood that some respondents who are not qualified to participate in the research will provide dishonest responses to demographic questions in order to be assured that they will be able to complete the survey and receive the results. This can skew the data.

The bottom line: For consumer research, offering to share the results is rarely an incentive for completing the survey. However, for certain business-to-business research projects, this can be an effective incentive.

See also:

• Online survey incentives - Guaranteed rewards
• Online survey incentives - Prize drawings

For online survey incentives, a common alternative to a guaranteed reward is a prize drawing or sweepstakes. Here are the pros and cons for this type of incentive.

Pros
This type of incentive offers several advantages over one-to-one incentives, including:

• Lower overall cost
• Ease of administration
• Little or no shipping expense
• Ability to increase the incentive effect by increasing the odds of winning

Another advantage of the drawing is that it limits the financial exposure of the research organization by setting a fixed cost for the incentive. Even if the number of respondents greatly exceeds expectations, the payout can remain the same.

Cons
If only one or two respondents are to be selected in the prize drawing, it is critically important to select a prize that appeals to potential respondents. For example, a drawing for a $10 gift card probably isn’t much of an incentive compared to a drawing for a $100 prepaid gas card.

Similarly, it is very important to establish an attractive odds of winning the drawing. In my experience, odds that tend to resonate with survey respondents fall in the 1-in-100 to 1-in-250 range.

Many experts caution against using this type of incentive for certain types of research, particularly projects in which respondents can be self-referred. For example, if a Web site has a banner inviting all visitors to take part in a survey and offering them a chance to win a prize for participating, it runs the risk of attracting “professional respondents” (people who complete the survey just to win the reward). The responses of such respondents are often dishonest.

The bottom line: Although there is no definitive research proving that prize drawings are always more effective than one-to-one incentives, it is logical to conclude that more times than not the chance to win a significant prize motivates more people than a guarantee to receive a cheap trinket or bauble.

See also:

• Online survey incentives - Guaranteed rewards
• Online survey incentives - Results sharing

Incentives offered to everyone who completes the survey are known as “one-to-one rewards” or “direct incentives.” Here are the pros and cons for this type of incentive.

Pros
The argument in favor of this type of incentive is that it increases the response rate because every respondent is rewarded for participating. When surveying professionals (doctors, attorneys, etc.), a fairly high cash reward for each respondent is generally the norm.

Cons
For consumer surveys, many projects lack a sufficient budget to offer an incentive that actually appeals to potential respondents. In the U.S., the average budget for a consumer incentive is approximately one dollar. Thus, the preponderance of stress balls, golf tees, etc.

Another problem with one-to-one incentives is the lack of an effective method for making small cash payments to large numbers of people. Many panel providers have worked around this problem by creating their own micropayment systems, usually based on respondents collecting points that can be exchanged for cash or merchandise.

The requirement to ship the incentive to each respondent significantly reduces the list of possible incentive items because only very lightweight items can be affordably mailed to respondents. As a rule, if the total cost for a one-to-one incentive (product cost plus shipping cost) exceeds one dollar, this approach generally loses its cost advantage.

Many experts caution against using incentives for some kinds of research, particularly projects in which respondents can be self-referred. For example, if a Web site has a banner inviting all visitors to take part in a survey and guaranteeing them a prize for participating, it runs the risk of attracting “professional respondents” (people who complete the survey just to win the reward). The responses of such respondents are often dishonest.

A final consideration with one-to-one incentives is that it can be important to take preventative measures to safeguard the budget from an unexpectedly high response rate. If the survey programming includes a limit on the number of interviews that will be completed, the survey is closed to respondents as soon as the quota is achieved.

The bottom line: one-to-one incentives can be effective at increasing response rates, particularly among high-income respondents. For consumer surveys, however, this type of incentive can be less effective.

See also:

• Online survey incentives - Prize drawings
• Online survey incentives - Results sharing

There is ample evidence that respondent incentives increase response rates for online surveys. What is less clear is what type of offer produces the best response.

The three most common types of respondent incentives for online surveys are:

1. One-to-one incentives: A small reward for everyone
2. Prize drawings: A significant reward for one or two people
3. Results sharing: A partial or complete report of results for everyone

Each type of incentive has a unique set of advantages, disadvantages, and applications. Each of the next three installments will examine one of the following:

1. Guaranteed rewards
2. Prize drawings
3. Results sharing

To make sure you receive this information, you can subscribe to this feed in the reader of your choice or bookmark the blog.

Many websites request visitors to take part in an online survey. These invitations generally fall into one of three formats: static text, banners, or pop-ups. Here are the pros and cons of using these types of invitations to find survey respondents.

These types of requests are inexpensive to administer but include two inherent drawbacks:

1. They tend to produce very low response rates
2. Because the sample of respondents is entirely self-selecting, the results are not representative of a larger audience (e.g., all customers)

Thus, although these invitations can provide useful feedback, they are not a good substitute for more traditional methods of locating survey respondents. Think of the information gleaned from this type of survey as qualitative/directional rather than quantitative/conclusive.

For more information about finding respondents, see these posts.

In addition to controlling access to online surveys, it is also important to ensure that your results are not skewed by respondents who did not answer questions seriously or truthfully. Here are a few tips to ensure that your final data contains quality responses from valid respondents.

The ever-increasing use of online surveys for market research has produced the unfortunate consequence of some survey respondents who are more interested in receiving a reward (incentive) for completing the survey than they are with providing accurate, thoughtful survey responses. These people are often referred to as “Professional Respondents.” Although they represent only a small minority of survey respondents, it only takes a few such people to skew your results.

Here are some methods you can use to reduce the odds of having respondents try to “game the system.”

1. Use a reputable panel with a strong (and published) set of verification standards. (For more information, see
   How to find respondents for your online survey (Part 2: Survey Panels)
2. Implement deterrents such as a legal/fraudulent access agreement that a respondent agree to comply with before they completing the survey.
3. Examine the data to identify fraudulent or irresponsible responses. For example, on a series of rating scale questions a pattern such as 7,7,7,7,7 or 1,2,3,4,5,1,2,3,4,5 may indicate this. Note that many full-featured survey systems (as opposed to the free or low-cost ones) use computer programs to identify and remove such responses.
4. Thoroughly check the final data (e.g., compare questions where the answers should not conflict, and review responses to open-ended questions).

The bottom line: a little extra effort to prevent fraudulent or irresponsible respondents from skewing your online survey data will significantly increase your confidence in the validity and accuracy of the results.

When conducting online surveys, researchers generally want to prevent unauthorized access by:

1. Restricting participation solely to those who were invited to respond.
2. Allowing each respondent to complete the survey only once.

Here is some practical advice for achieving these important objectives.

(more…)

Online surveys can be configured so a respondent is required to provide a response to a question before proceeding to the next question. This practice is known as forced completion. Here are the arguments for and against this practice.

(more…)

In Part 1 of this 3-part series, we learned that 87 percent of online surveys conducted in the United States in 2006 were deployed using a preexisting survey panel. In Part 2, we explored why panels are the preferred method for online interviewing and how to select a panel provider for a market research project.

In this final installment, we will discuss the pros and cons of another popular method of recruiting respondents for online surveys: Customer Lists.

(more…)

In Part 1 of this 3-part series, we learned that 87 percent of online surveys conducted in the United States in 2006 were deployed using a preexisting survey panel. In this installment, we will explore why panels are the preferred method for online interviewing and how to select a panel provider for a market research project.

(more…)