By Barbara Buell
STANFORD GRADUATE SCHOOL OF BUSINESS—YOU’VE JUST DROPPED YOUR CAR OFF at the dealership for service and a manager politely informs you that you will be asked to answer a quality survey by phone after the work is completed. On another occasion, you’ve just walked out of the supermarket and a store employee unexpectedly grills you about the convenience and speed of service you just received. In which of the two situations are you likely to be more critical? Hands down: You found more faults at the dealership.
No, not because some drivers are skeptical about service pricing at their dealerships. New consumer research shows that when people are told in advance that they will be asked to respond to a survey, they look for problems more actively. Research recently completed by marketing professor Itamar Simonson and coauthor Chezy Ofir, a marketing professor at Hebrew University in Jerusalem, is quite relevant in this increasingly survey-happy world. Companies want to satisfy their customers, so they are issuing more and more quality polls. “Nowadays, consumers, voters, and business customers are constantly asked to evaluate products, services, people, politicians, and proposals,” says Simonson. “One hundred years ago people were not asked to evaluate as much.”
The researchers found that people who expect to evaluate are decidedly more negative. They also discovered that merely asking people to state their expectations before they receive a service made people morenegative—even though their predispositions may have been quite positive. For example, people who are asked if they think they will like a movie before seeing it will be statistically more negative than people who were never asked that question.
Simonson and Ofir studied the responses of customers who had called for service at a major computer hardware and software company. The researchers divided the customers into four groups. Participants in the first group were told a technician would service their problems and that they would subsequently be asked about the service, such as whether the tech was on time, whether the employee was polite, and whether he or she solved the problem. A second group was not told there would be an evaluation, but the customers were asked to state their expectations, such as how long they thought it would take for a tech to arrive. A third group was told both: to state their expectations and to expect a survey. Members of a control group knew nothing but were later polled.
The result: People who expected to evaluate were significantly more negative than members of the control group. The same was true of the group asked to state their expectations ahead of time. Interestingly, the group that was the most dissatisfied was the one that was asked their expectations and also warned about a survey.
A couple of things are going on here, say the researchers. When people expect to evaluate, they are paying more attention because they know they will have to report later. “And we know when people pay more attention, they tend to notice and remember best negative things,” says Simonson. “After all, if things go as you expect, that’s a nonevent.” In addition, “if you say everything is fine, you feel you haven’t done your job,” he says. “I am looking for negative things so I have something to say.”
The researchers replicated the study at an electric utility, a supermarket, a television program screening, and even at a zoo, where visitors were asked how they liked it after they exited. Results were the same.
The researchers warn that while marketers must stay on top of customer desires and complaints, they must also be aware of the effects the mere expectation of filling out a survey can have on how customers view their experience. “It may not be realistic,” says Simonson. “They may be chronically more negative, pointing out problems that are not problems to the average consumer,” he says. “You want people who are representative of the marketplace.
Businesses must recognize the limitations and biases a survey itself can introduce. One solution may be to rotate surveys through clusters of customers so that the same consumers are not constantly responding
And when possible, suggests Simonson, try not to warn people in advance that they will be polled—you get more natural feedback. In some cases, sheltering evaluators may be unavoidable, since the more common surveys become, the more people come to expect them, a phenomenon that seems likely to increase with the ease of customer communication and personalization on the Internet.
The Effect of Expecting to Evaluate on Quality and Satisfaction Evaluations, Chezy Ofir and Itamar Simonson, GSB Research Paper #1608, July 1999
Chip Heath, Ideas that Stick Without Advertising Dollars
Chip Heath Faculty Profile
Jennifer L. Aaker, Matching the Pitch to the Perspective
Jennifer L. Aaker Faculty Profile
Sonya Grier, Selling Violence to Children
Itamar Simonson, The Limits of One to One Marketing
Itamar Simonson Faculty Profile
Also on Stanford Knowledgebase: