Measure customer use of things, not customer satisfaction with things

“The problem with customer satisfaction surveys,” a sales manager once told me, “is that they don’t predict future behavior. We’ve found that customers who are ‘satisfied’ can be as likely to leave us as those who say they are not satisfied.”

When we measured a particular website in relation to how able customers were to complete their top tasks, the results were dismal. The marketing manager, on the other hand, wasn’t worried at all. “Our customer satisfaction results are fine,” she said. “That’s what we go by around here.”

Over the years, we’ve seen customer satisfaction data from a range of organizations in multiple industries and countries. The numbers are all very high, but they bear practically no resemblance to task performance on the sites and apps. There can be anything from a 30%-50% difference between task success and stated customer satisfaction. For example, you could have a totally wonderful 80% customer satisfaction rating and an absolutely terrible 30% task success rate.

The Net Promoter Score was developed to address some of these problems. In 2017, Jared Spool wrote an excellent article on the many flaws with this method. “It doesn’t help businesses grow. It doesn’t even tell the management how loyal the customer is,” Jared writes.

The problem with customer satisfaction surveys is as old as the hills. What people say and what they do can be very different things. Furthermore, people tend not to be very good at predicting what they are going to do in the future.

If we want good metrics, then we need to measure the actual customer experience. It won’t solve everything but it’s a lot better than asking people how satisfied they are. Our relationship with organizations is complicated. I’m an Amazon customer. However, I’m actively trying to buy as little as possible from them because I think they treat their staff appallingly, and also because I think they’re becoming a monopoly.

However, when I recently had a problem with an Amazon delivery, the support I got was excellent. Any issues (and there haven’t been many) that I have had with Amazon over the years have been addressed quickly. I never get the sense that they’re trying to rip me off, either—a sense I get from a huge range of other ‘brands’ I have to deal with. So, it’s very hard to leave Amazon entirely because they make things so easy and convenient.

Not like PayPal. I had an issue with a credit card with them and it’s been a nightmare; such a waste of time. An automated phone system that makes you want to jump from the nearest bridge. Email support that never gets back to you. A Twitter response that uses lovely language but doesn’t solve the problem. If I could never use PayPal again, I’d do so. And I have always found PayPal absolutely terrible to deal with.

Saving time, simplicity, ease-of-use, convenience. These are the things we need to measure, not ask people their opinion on. Measure, observe. One of the biggest competitive advantages you can have today is ease-of-use. How do you measure ease-of-use? Measure use.

Is it easy to measure use? No. Is it easy to measure ‘customer satisfaction’? Yes. Lazy, ‘cost conscious’ organizations like to find what’s easy to measure and then they manage that. You must decide what is vital to manage and then measure that. In a digital economy, the most vital thing to manage is use.

Net Promoter Score Considered Harmful: Jared Spool