Todd Hollander Market Research

I recently wrote about a frustrating experience trying to place an order with Comcast for high speed internet and digital cable television (”Ordering from Comcast - Like a visit to the proctologist” - March 12, 2008). Within 24 hours of that post, I received a call from the Comcast corporate office apologizing for the problem and seeking to resolve it (”Comcast admits mistake” - March 18, 2008). Yesterday, the company installed the new services and waived one of the fees.

Here’s what I learned from the experience:

• This blog afforded me an opportunity unavailable to most disgruntled customers: a bully pulpit to publicly declare my dissatisfaction and win the attention of someone who could do something about it.
• When dissatisfied as a customer, I should always consider taking action rather than suffering in silence.
• Although Comcast has much room for improvement in delivering a customer-centric order/upgrade experience, in my case they were willing to acknowledge the problem and take effective steps to remedy it.
• High-speed cable internet access is much faster than DSL. As in: lightning fast ; no comparison; way better.

So at least for today, I have blazing internet access and on-demand television. Life is good. Maybe even “comcastic.”

In a recent post, I wrote about a terrible experience I had trying to place an order with Comcast for high speed internet and digital cable television. Within 24 hours of publishing that report, I received a call from the Comcast corporate office. The caller said he saw the blog entry, apologized for the problem, and offered his assistance to resolve it. He took some information and told me that I would receive a call from the local Comcast office shortly.

The next day I received the call from the local office. The woman identified herself as Mrs. Callahan. She did this twice, never offering a first name. Maybe its just me, but I find this form of address stilted and off-putting, particularly when the addressee is attempting to right a wrong. Nonetheless, Mrs. Callahan gathered some info and promised to call me back in a few minutes. When she called back, she offered to waive the $17.95 “upgrade fee” for my order and offered an installation date two business days later (a full week sooner than the earliest date offered when I tried to order the installation online).

Today is that day. The time: between 11:00 and 2:00. I’ll keep you posted.

Today I tried to place an order with Comcast for high speed internet and digital cable television. What should have been a simple transaction was anything but easy. Here’s how it went…

1. As an existing customer, I logged into the Comcast website using my username and password.
2. After negotiating a fairly cumbersome order process (selecting a service bundle, leasing a modem, and scheduling installation), I was taken to a live chat where a Customer Service Representative was to complete my transaction.
3. The chat session did not support my Firefox web browser, so having already invested 10 to 15-minutes into the ordering process…
4. I had to redo the entire transaction with another browser.
5. I was again taken to the live chat session to complete the transaction. Here is the actual transcript from the chat:

chat id : 7ef71f40-5d5b-4183-b670-467969b02ba3

Problem : Order Information

Arnel.24970 > Hello Todd_, Thank you for contacting Comcast Live Chat Support. My name is Arnel.24970. Please give me one moment to review your information.

Arnel.24970 > Welcome to Comcast. We are very happy to provide service for you.

Arnel.24970 > Todd, are you a new or existing Comcast customer?

Todd_ > existing

Arnel.24970 > Okay great.

Arnel.24970 > MayI have your account number please?

Arnel.24970 > I do apologize for the misspelled word. I also accidentally hit the ENTER key.

Todd_ > I don’t have it with me. Can you look it up by address?

Arnel.24970 > No problem.

Arnel.24970 > May I know what services do you currently have with Comcast?

Todd_ > tv only - expanded package I think

Arnel.24970 > Okay.

Arnel.24970 > I will be very happy to process your order today.

Arnel.24970 > Let me go ahead and check out your order first.

Arnel.24970 > Thank you for waiting.

Arnel.24970 > Todd, can you please hold for one moment while I try to locate your address in the database?

Todd_ > yes

Arnel.24970 > Okay.

Arnel.24970 > One moment please.

Arnel.24970 > Thank you for waiting.

Arnel.24970 > Can you please provide me your Social Security Number and your active e-mail address?

Todd_ > the email address for the existing account?

Arnel.24970 > Yes please.

Todd_ > [provided email address]

Arnel.24970 > Thank you.

Arnel.24970 > For account identification can you please verify the amount and date of the last payment made to the account?

Todd_ > I have no idea. My wife pays the bill.

Arnel.24970 > I do apologize, Todd. We will be unable to process your order without proper account verification.

Arnel.24970 > Please do chat us back once you have that information.

Arnel.24970 > Were you able to receive my last message?

Todd_ > so I have wasted 30 minutes?

Arnel.24970 > I do apologize for the inconvenience this has caused you.

Todd_ > do you have the number for bellsouth? Maybe they are easier to do business with.

Arnel.24970 > For directions and the contact information of the local office in your area please visit:

https://www.comcast.com/Localization/Localize.ashx

Arnel.24970 > Is there anything else that I can help you with?

Arnel.24970 > Were you able to receive my last message?

Todd_ > If I wanted to place this order, would it still be in my cart or would I have to start all over?

Arnel.24970 > It is possible. Just provide your order I.D. to the representative. That would be [order #].

Todd_ > Gee, glad I asked. Is there no other way to verify the account except to provide the amount and date of the last payment?

Arnel.24970 > I do apologize, Todd Base on the information listed on your account, the date and the last payment made is the only information to verify.

Todd_ > Seems like I should have been told that I would need that info before spending all this time placing the order and scheduling installation.

Arnel.24970 > I really do apologize.

Arnel.24970 > Please do chat us back once you have that information.

Here is what I find so dissatisfying about this experience:

1. I should have been informed that in order to complete the order I would be required to provide certain information to verify the account.
2. Even though I had logged into my account to place the order, the agent…

• Did not know that I was an existing customer
• Did not know my account number.
• Did not know what services I was being provided
• Asked for my Social Security Number
• Asked for my email address

The bottom line: The word “comcastic” should be defined as: “Invasive, painful, frustrating, and dissatisfying.” Example: “My visit to the proctologist was utterly comcastic.”

As an incentive for online survey participation, some companies offer to share the survey results with respondents. Here are the pros and cons for this type of incentive.

Pros
Offering to share some or all of the survey results with respondents can sometimes be effective with business-to-business research. For employees of certain companies, incentives such as guaranteed rewards and prize drawings are considered conflicts of interest and are therefore not permitted. Thus, the ability to share in the insight gleaned from the survey can be an extra incentive.

Cons
For consumer research, offering to share the results with respondents is rarely an effective incentive because most consumers prefer guaranteed rewards or prize drawings.

For business-to-business research, offering to share the results can increase the likelihood that some respondents who are not qualified to participate in the research will provide dishonest responses to demographic questions in order to be assured that they will be able to complete the survey and receive the results. This can skew the data.

The bottom line: For consumer research, offering to share the results is rarely an incentive for completing the survey. However, for certain business-to-business research projects, this can be an effective incentive.

See also:

• Online survey incentives - Guaranteed rewards
• Online survey incentives - Prize drawings

For online survey incentives, a common alternative to a guaranteed reward is a prize drawing or sweepstakes. Here are the pros and cons for this type of incentive.

Pros
This type of incentive offers several advantages over one-to-one incentives, including:

• Lower overall cost
• Ease of administration
• Little or no shipping expense
• Ability to increase the incentive effect by increasing the odds of winning

Another advantage of the drawing is that it limits the financial exposure of the research organization by setting a fixed cost for the incentive. Even if the number of respondents greatly exceeds expectations, the payout can remain the same.

Cons
If only one or two respondents are to be selected in the prize drawing, it is critically important to select a prize that appeals to potential respondents. For example, a drawing for a $10 gift card probably isn’t much of an incentive compared to a drawing for a $100 prepaid gas card.

Similarly, it is very important to establish an attractive odds of winning the drawing. In my experience, odds that tend to resonate with survey respondents fall in the 1-in-100 to 1-in-250 range.

Many experts caution against using this type of incentive for certain types of research, particularly projects in which respondents can be self-referred. For example, if a Web site has a banner inviting all visitors to take part in a survey and offering them a chance to win a prize for participating, it runs the risk of attracting “professional respondents” (people who complete the survey just to win the reward). The responses of such respondents are often dishonest.

The bottom line: Although there is no definitive research proving that prize drawings are always more effective than one-to-one incentives, it is logical to conclude that more times than not the chance to win a significant prize motivates more people than a guarantee to receive a cheap trinket or bauble.

See also:

• Online survey incentives - Guaranteed rewards
• Online survey incentives - Results sharing

Incentives offered to everyone who completes the survey are known as “one-to-one rewards” or “direct incentives.” Here are the pros and cons for this type of incentive.

Pros
The argument in favor of this type of incentive is that it increases the response rate because every respondent is rewarded for participating. When surveying professionals (doctors, attorneys, etc.), a fairly high cash reward for each respondent is generally the norm.

Cons
For consumer surveys, many projects lack a sufficient budget to offer an incentive that actually appeals to potential respondents. In the U.S., the average budget for a consumer incentive is approximately one dollar. Thus, the preponderance of stress balls, golf tees, etc.

Another problem with one-to-one incentives is the lack of an effective method for making small cash payments to large numbers of people. Many panel providers have worked around this problem by creating their own micropayment systems, usually based on respondents collecting points that can be exchanged for cash or merchandise.

The requirement to ship the incentive to each respondent significantly reduces the list of possible incentive items because only very lightweight items can be affordably mailed to respondents. As a rule, if the total cost for a one-to-one incentive (product cost plus shipping cost) exceeds one dollar, this approach generally loses its cost advantage.

Many experts caution against using incentives for some kinds of research, particularly projects in which respondents can be self-referred. For example, if a Web site has a banner inviting all visitors to take part in a survey and guaranteeing them a prize for participating, it runs the risk of attracting “professional respondents” (people who complete the survey just to win the reward). The responses of such respondents are often dishonest.

A final consideration with one-to-one incentives is that it can be important to take preventative measures to safeguard the budget from an unexpectedly high response rate. If the survey programming includes a limit on the number of interviews that will be completed, the survey is closed to respondents as soon as the quota is achieved.

The bottom line: one-to-one incentives can be effective at increasing response rates, particularly among high-income respondents. For consumer surveys, however, this type of incentive can be less effective.

See also:

• Online survey incentives - Prize drawings
• Online survey incentives - Results sharing

There is ample evidence that respondent incentives increase response rates for online surveys. What is less clear is what type of offer produces the best response.

The three most common types of respondent incentives for online surveys are:

1. One-to-one incentives: A small reward for everyone
2. Prize drawings: A significant reward for one or two people
3. Results sharing: A partial or complete report of results for everyone

Each type of incentive has a unique set of advantages, disadvantages, and applications. Each of the next three installments will examine one of the following:

1. Guaranteed rewards
2. Prize drawings
3. Results sharing

To make sure you receive this information, you can subscribe to this feed in the reader of your choice or bookmark the blog.

Many websites request visitors to take part in an online survey. These invitations generally fall into one of three formats: static text, banners, or pop-ups. Here are the pros and cons of using these types of invitations to find survey respondents.

These types of requests are inexpensive to administer but include two inherent drawbacks:

1. They tend to produce very low response rates
2. Because the sample of respondents is entirely self-selecting, the results are not representative of a larger audience (e.g., all customers)

Thus, although these invitations can provide useful feedback, they are not a good substitute for more traditional methods of locating survey respondents. Think of the information gleaned from this type of survey as qualitative/directional rather than quantitative/conclusive.

For more information about finding respondents, see these posts.

A common mistake made by those conducting online surveys is failing to send reminder emails to potential respondents. Another common mistake is to send out too many reminders. Here is some advice that will help you develop an effective reminder system.

Failing to send reminder emails to potential survey participants can result in unacceptable response rates. Conversely, annoying respondents by sending too many reminders can adversely affect both the response rate and the quality of the data collected. Therefore, it is important to send the right number of reminders with the right wording at the right time.

One of the reasons some researchers fail to send reminders is that they think the reminder should be sent only to those who have not yet completed the survey. The problem is they don’t know how to do it. Although it is possible to identify those who have already completed the survey, the tactics for doing this are risky and can be viewed as underhanded (see “How to calculate response rates for online surveys“).

The good news is that you can send an email reminder to everyone on the mailing list without annoying those who have already completed the survey. Here’s how…

1. Forward the original invitation
2. Change the Subject Line to: Reminder: [original invitation subject line]
3. Add this text at the top of the message: If you have already completed the survey, thank you. If you have not, please see below.

That’s all you need to do. Nine times out of 10, one reminder at the mid-point of the data collection period does the trick. Occasionally, a second reminder is needed. Important: Never send more than two reminder emails - it will only annoy people.

The bottom line: Some people will take your survey as soon as they receive the invitation email. Others will need a follow-up email reminder. One reminder at the mid-point of the data collection period is best. Never send more than two reminder emails.

Calculating the response rate for an online survey is an important but sometimes difficult process. Here are some tips to help you calculate an accurate response rate.

For surveys in which potential respondents are invited by email, the traditional practice of calculating response rate as the number of completed interviews divided by the number of invitations generally understates the response rate because it assumes that all invitation were received and opened. Because of the prevalence of spam and spam filters, and the simple fact that many people are inundated with email messages, a more accurate way of measuring response rate is needed.

A truer measure of response rate is calculated as the number of completed interviews divided by a) the number of invitations sent minus b) the number of bounce backs. However, even this calculation can understate the response rate because:

1. Not all failed emails generate a bounce back
2. It still assumes that all emails that were received were seen by the recipients

Although it is possible to count the number of email invitations that were seen (not just received, but opened), the process by which this is done can be detrimental to the study.

Return Receipts

The option of requesting a return receipt is included in many email clients, but is not a good option because it is so intrusive. Generally, when a sender requests a return receipt the recipient’s email client posts a message such as: “This user has requested a return receipt. Allow? Yes/No”. If the recipient clicks “No” (and many do as a standard response), that is the end of that.

Using an Image

This method is more effective but a lot sneakier. In this process, the survey invitation is formatted in HTML and includes a tiny image tag (often only one pixel) such as:

<img src='http://yoursite.com/UserOpenedEmail.aspx?UID=username@domain.com'>

As soon as the email is opened or displayed in the preview pane, the website receives a page request and extracts the email address from the querystring to confirm that the email sent to that address was opened.

Sounds great, right? Well, maybe not. One problem is that not all recipients receive email in HTML format (many use text only). Another problem is that many email clients now default to not automatically downloading images. But the biggest problem is that this method seems very underhanded. If potential recipients discover that you are doing it, they are likely to feel you are trying to deceive them. As a result, they are less inclined to complete the survey or provide frank responses.

The bottom line: the best practice for calculating the response rate is to divide the number of completed interviews by the number of invitations sent minus the number of bounce backs. Although it is possible to achieve a more accurate calculation, the risks of doing so generally outweigh the benefits.