Todd Hollander Market Research

I recently wrote about a frustrating experience trying to place an order with Comcast for high speed internet and digital cable television (”Ordering from Comcast - Like a visit to the proctologist” - March 12, 2008). Within 24 hours of that post, I received a call from the Comcast corporate office apologizing for the problem and seeking to resolve it (”Comcast admits mistake” - March 18, 2008). Yesterday, the company installed the new services and waived one of the fees.

Here’s what I learned from the experience:

• This blog afforded me an opportunity unavailable to most disgruntled customers: a bully pulpit to publicly declare my dissatisfaction and win the attention of someone who could do something about it.
• When dissatisfied as a customer, I should always consider taking action rather than suffering in silence.
• Although Comcast has much room for improvement in delivering a customer-centric order/upgrade experience, in my case they were willing to acknowledge the problem and take effective steps to remedy it.
• High-speed cable internet access is much faster than DSL. As in: lightning fast ; no comparison; way better.

So at least for today, I have blazing internet access and on-demand television. Life is good. Maybe even “comcastic.”

In a recent post, I wrote about a terrible experience I had trying to place an order with Comcast for high speed internet and digital cable television. Within 24 hours of publishing that report, I received a call from the Comcast corporate office. The caller said he saw the blog entry, apologized for the problem, and offered his assistance to resolve it. He took some information and told me that I would receive a call from the local Comcast office shortly.

The next day I received the call from the local office. The woman identified herself as Mrs. Callahan. She did this twice, never offering a first name. Maybe its just me, but I find this form of address stilted and off-putting, particularly when the addressee is attempting to right a wrong. Nonetheless, Mrs. Callahan gathered some info and promised to call me back in a few minutes. When she called back, she offered to waive the $17.95 “upgrade fee” for my order and offered an installation date two business days later (a full week sooner than the earliest date offered when I tried to order the installation online).

Today is that day. The time: between 11:00 and 2:00. I’ll keep you posted.

Today I tried to place an order with Comcast for high speed internet and digital cable television. What should have been a simple transaction was anything but easy. Here’s how it went…

1. As an existing customer, I logged into the Comcast website using my username and password.
2. After negotiating a fairly cumbersome order process (selecting a service bundle, leasing a modem, and scheduling installation), I was taken to a live chat where a Customer Service Representative was to complete my transaction.
3. The chat session did not support my Firefox web browser, so having already invested 10 to 15-minutes into the ordering process…
4. I had to redo the entire transaction with another browser.
5. I was again taken to the live chat session to complete the transaction. Here is the actual transcript from the chat:

chat id : 7ef71f40-5d5b-4183-b670-467969b02ba3

Problem : Order Information

Arnel.24970 > Hello Todd_, Thank you for contacting Comcast Live Chat Support. My name is Arnel.24970. Please give me one moment to review your information.

Arnel.24970 > Welcome to Comcast. We are very happy to provide service for you.

Arnel.24970 > Todd, are you a new or existing Comcast customer?

Todd_ > existing

Arnel.24970 > Okay great.

Arnel.24970 > MayI have your account number please?

Arnel.24970 > I do apologize for the misspelled word. I also accidentally hit the ENTER key.

Todd_ > I don’t have it with me. Can you look it up by address?

Arnel.24970 > No problem.

Arnel.24970 > May I know what services do you currently have with Comcast?

Todd_ > tv only - expanded package I think

Arnel.24970 > Okay.

Arnel.24970 > I will be very happy to process your order today.

Arnel.24970 > Let me go ahead and check out your order first.

Arnel.24970 > Thank you for waiting.

Arnel.24970 > Todd, can you please hold for one moment while I try to locate your address in the database?

Todd_ > yes

Arnel.24970 > Okay.

Arnel.24970 > One moment please.

Arnel.24970 > Thank you for waiting.

Arnel.24970 > Can you please provide me your Social Security Number and your active e-mail address?

Todd_ > the email address for the existing account?

Arnel.24970 > Yes please.

Todd_ > [provided email address]

Arnel.24970 > Thank you.

Arnel.24970 > For account identification can you please verify the amount and date of the last payment made to the account?

Todd_ > I have no idea. My wife pays the bill.

Arnel.24970 > I do apologize, Todd. We will be unable to process your order without proper account verification.

Arnel.24970 > Please do chat us back once you have that information.

Arnel.24970 > Were you able to receive my last message?

Todd_ > so I have wasted 30 minutes?

Arnel.24970 > I do apologize for the inconvenience this has caused you.

Todd_ > do you have the number for bellsouth? Maybe they are easier to do business with.

Arnel.24970 > For directions and the contact information of the local office in your area please visit:

https://www.comcast.com/Localization/Localize.ashx

Arnel.24970 > Is there anything else that I can help you with?

Arnel.24970 > Were you able to receive my last message?

Todd_ > If I wanted to place this order, would it still be in my cart or would I have to start all over?

Arnel.24970 > It is possible. Just provide your order I.D. to the representative. That would be [order #].

Todd_ > Gee, glad I asked. Is there no other way to verify the account except to provide the amount and date of the last payment?

Arnel.24970 > I do apologize, Todd Base on the information listed on your account, the date and the last payment made is the only information to verify.

Todd_ > Seems like I should have been told that I would need that info before spending all this time placing the order and scheduling installation.

Arnel.24970 > I really do apologize.

Arnel.24970 > Please do chat us back once you have that information.

Here is what I find so dissatisfying about this experience:

1. I should have been informed that in order to complete the order I would be required to provide certain information to verify the account.
2. Even though I had logged into my account to place the order, the agent…

• Did not know that I was an existing customer
• Did not know my account number.
• Did not know what services I was being provided
• Asked for my Social Security Number
• Asked for my email address

The bottom line: The word “comcastic” should be defined as: “Invasive, painful, frustrating, and dissatisfying.” Example: “My visit to the proctologist was utterly comcastic.”

As an incentive for online survey participation, some companies offer to share the survey results with respondents. Here are the pros and cons for this type of incentive.

Pros
Offering to share some or all of the survey results with respondents can sometimes be effective with business-to-business research. For employees of certain companies, incentives such as guaranteed rewards and prize drawings are considered conflicts of interest and are therefore not permitted. Thus, the ability to share in the insight gleaned from the survey can be an extra incentive.

Cons
For consumer research, offering to share the results with respondents is rarely an effective incentive because most consumers prefer guaranteed rewards or prize drawings.

For business-to-business research, offering to share the results can increase the likelihood that some respondents who are not qualified to participate in the research will provide dishonest responses to demographic questions in order to be assured that they will be able to complete the survey and receive the results. This can skew the data.

The bottom line: For consumer research, offering to share the results is rarely an incentive for completing the survey. However, for certain business-to-business research projects, this can be an effective incentive.

See also:

• Online survey incentives - Guaranteed rewards
• Online survey incentives - Prize drawings

For online survey incentives, a common alternative to a guaranteed reward is a prize drawing or sweepstakes. Here are the pros and cons for this type of incentive.

Pros
This type of incentive offers several advantages over one-to-one incentives, including:

• Lower overall cost
• Ease of administration
• Little or no shipping expense
• Ability to increase the incentive effect by increasing the odds of winning

Another advantage of the drawing is that it limits the financial exposure of the research organization by setting a fixed cost for the incentive. Even if the number of respondents greatly exceeds expectations, the payout can remain the same.

Cons
If only one or two respondents are to be selected in the prize drawing, it is critically important to select a prize that appeals to potential respondents. For example, a drawing for a $10 gift card probably isn’t much of an incentive compared to a drawing for a $100 prepaid gas card.

Similarly, it is very important to establish an attractive odds of winning the drawing. In my experience, odds that tend to resonate with survey respondents fall in the 1-in-100 to 1-in-250 range.

Many experts caution against using this type of incentive for certain types of research, particularly projects in which respondents can be self-referred. For example, if a Web site has a banner inviting all visitors to take part in a survey and offering them a chance to win a prize for participating, it runs the risk of attracting “professional respondents” (people who complete the survey just to win the reward). The responses of such respondents are often dishonest.

The bottom line: Although there is no definitive research proving that prize drawings are always more effective than one-to-one incentives, it is logical to conclude that more times than not the chance to win a significant prize motivates more people than a guarantee to receive a cheap trinket or bauble.

See also:

• Online survey incentives - Guaranteed rewards
• Online survey incentives - Results sharing

Incentives offered to everyone who completes the survey are known as “one-to-one rewards” or “direct incentives.” Here are the pros and cons for this type of incentive.

Pros
The argument in favor of this type of incentive is that it increases the response rate because every respondent is rewarded for participating. When surveying professionals (doctors, attorneys, etc.), a fairly high cash reward for each respondent is generally the norm.

Cons
For consumer surveys, many projects lack a sufficient budget to offer an incentive that actually appeals to potential respondents. In the U.S., the average budget for a consumer incentive is approximately one dollar. Thus, the preponderance of stress balls, golf tees, etc.

Another problem with one-to-one incentives is the lack of an effective method for making small cash payments to large numbers of people. Many panel providers have worked around this problem by creating their own micropayment systems, usually based on respondents collecting points that can be exchanged for cash or merchandise.

The requirement to ship the incentive to each respondent significantly reduces the list of possible incentive items because only very lightweight items can be affordably mailed to respondents. As a rule, if the total cost for a one-to-one incentive (product cost plus shipping cost) exceeds one dollar, this approach generally loses its cost advantage.

Many experts caution against using incentives for some kinds of research, particularly projects in which respondents can be self-referred. For example, if a Web site has a banner inviting all visitors to take part in a survey and guaranteeing them a prize for participating, it runs the risk of attracting “professional respondents” (people who complete the survey just to win the reward). The responses of such respondents are often dishonest.

A final consideration with one-to-one incentives is that it can be important to take preventative measures to safeguard the budget from an unexpectedly high response rate. If the survey programming includes a limit on the number of interviews that will be completed, the survey is closed to respondents as soon as the quota is achieved.

The bottom line: one-to-one incentives can be effective at increasing response rates, particularly among high-income respondents. For consumer surveys, however, this type of incentive can be less effective.

See also:

• Online survey incentives - Prize drawings
• Online survey incentives - Results sharing

There is ample evidence that respondent incentives increase response rates for online surveys. What is less clear is what type of offer produces the best response.

The three most common types of respondent incentives for online surveys are:

1. One-to-one incentives: A small reward for everyone
2. Prize drawings: A significant reward for one or two people
3. Results sharing: A partial or complete report of results for everyone

Each type of incentive has a unique set of advantages, disadvantages, and applications. Each of the next three installments will examine one of the following:

1. Guaranteed rewards
2. Prize drawings
3. Results sharing

To make sure you receive this information, you can subscribe to this feed in the reader of your choice or bookmark the blog.

Many websites request visitors to take part in an online survey. These invitations generally fall into one of three formats: static text, banners, or pop-ups. Here are the pros and cons of using these types of invitations to find survey respondents.

These types of requests are inexpensive to administer but include two inherent drawbacks:

1. They tend to produce very low response rates
2. Because the sample of respondents is entirely self-selecting, the results are not representative of a larger audience (e.g., all customers)

Thus, although these invitations can provide useful feedback, they are not a good substitute for more traditional methods of locating survey respondents. Think of the information gleaned from this type of survey as qualitative/directional rather than quantitative/conclusive.

For more information about finding respondents, see these posts.

In addition to controlling access to online surveys, it is also important to ensure that your results are not skewed by respondents who did not answer questions seriously or truthfully. Here are a few tips to ensure that your final data contains quality responses from valid respondents.

The ever-increasing use of online surveys for market research has produced the unfortunate consequence of some survey respondents who are more interested in receiving a reward (incentive) for completing the survey than they are with providing accurate, thoughtful survey responses. These people are often referred to as “Professional Respondents.” Although they represent only a small minority of survey respondents, it only takes a few such people to skew your results.

Here are some methods you can use to reduce the odds of having respondents try to “game the system.”

1. Use a reputable panel with a strong (and published) set of verification standards. (For more information, see
   How to find respondents for your online survey (Part 2: Survey Panels)
2. Implement deterrents such as a legal/fraudulent access agreement that a respondent agree to comply with before they completing the survey.
3. Examine the data to identify fraudulent or irresponsible responses. For example, on a series of rating scale questions a pattern such as 7,7,7,7,7 or 1,2,3,4,5,1,2,3,4,5 may indicate this. Note that many full-featured survey systems (as opposed to the free or low-cost ones) use computer programs to identify and remove such responses.
4. Thoroughly check the final data (e.g., compare questions where the answers should not conflict, and review responses to open-ended questions).

The bottom line: a little extra effort to prevent fraudulent or irresponsible respondents from skewing your online survey data will significantly increase your confidence in the validity and accuracy of the results.

When conducting online surveys, researchers generally want to prevent unauthorized access by:

1. Restricting participation solely to those who were invited to respond.
2. Allowing each respondent to complete the survey only once.

Here is some practical advice for achieving these important objectives.

(more…)