The trouble with magic metrics

Executives like the idea of using a single magic metric to evaluate customer service because it’s so simple. “Do well at this,” the thinking goes, “and you’re doing well.”

Unfortunately, the real magic happens when you start peeling back the layers of your data to find out what’s really going on. If you look carefully at so-called single score metrics like the Net Promoter System, you’ll realize the score is just a starting point for evaluation. It’s the underlying analysis and continuous drive to improve that’s really important.

For example, a few months ago I blogged about my experience with Verizon’s Global Traveler Program. I’d rate my likelihood to recommend their service as a 7 on a scale of 1 to 10, but this rating doesn’t reveal that my score is really a sum of good, great, and poor experiences. A closer look at experiences like mine is needed for Verizon to clearly understand how to make their service truly exceptional while cutting down on costly calls to technical support.

Not All Promoters Are Loyal
I really enjoy reading the Wall Street Journal. Their informative articles help me stay on top of the news I care about while their style really captures my attention. I follow them on Twitter to get the latest updates on their news coverage and I consider it a plus when I stay at a hotel that provides free copies to their guests.

So why did I recently cancel my subscription?

Simply put, subscribing to the Wall Street Journal had become a hassle. You could classify me as a “Promoter” if you used a Net Promoter System because I would certainly recommend that you read their news coverage. However, I’d give the Wall Street Journal low marks on the amount of effort required to subscribe to them via my iPad.

Customer Effort Score
One of the latest magic metrics to gain popularity is called the Customer Effort Score. It essentially gauges how easy or difficult it is to do business with a company. According to research published by the Corporate Executive Board, Customer Effort is a better predictor of loyalty than a Net Promoter Score.

Reducing effort is where the Wall Street Journal could have saved my business. I subscribed via my iPad because I liked the promise of reading the paper wherever I went and getting updated content throughout the day. However, they frequently released updates to their iPad app, which meant that I couldn’t read the paper until I performed the update. This got more and more annoying until I finally decided the price of the subscription wasn’t worth the extra effort.

Ironically, cancelling my subscription also required more effort than necessary since the only way to cancel a subscription is to call. Their website states this is for security reasons, but the truth is they want to take one more shot at keeping your business by making you speak to a retention specialist.

So, just make things easy, right?!
Not so fast. Let’s compare one of my favorite examples, McDonalds vs. In-N-Out Burger. Getting a burger at McDonalds is relatively easy. The drive-through line moves pretty quickly and most orders placed at the counter are delivered almost instantly. On the other hand, there’s always a wait to get your food at In-N-Out.

If we judged the two by Customer Effort alone, McDonalds would win hands down. The problem with that assumption is that you’d then be stuck with a McDonalds hamburger which consistently scores poorly in consumer taste tests. As legions of devoted In-N-Out fans will tell you, getting a tasty In-N-Out burger served by a cheerful employee in a clean store is absolutely worth the extra effort.

There Are Three Kinds of Lies
One of my favorite quotes is from Mark Twain:

“There are three kinds of lies: lies, damned lies, and statistics.”

My take-away is data can be very useful, but it can also obscure the truth. Customer service metrics like Net Promoter Score or Customer Effort Score are fantastically useful tools, but achieving a certain score should never be the ultimate goal.

If all you want to do is achieve a certain Net Promoter Score, I can show you a dozen tricks to make that happen. For instance, one enterprising store manager at the Gap offered a 20% discount in exchange for a 10 on their Net Promoter survey. That will certainly improve the score but not necessarily the service.

The most successful companies first ask, “How can we do better” and then find the right data to help them continuously improve.

The hidden dark side of good service

I recently traveled overseas and had to get a loaner phone from Verizon since my iPhone 4 wouldn't work in the UK or Ireland. If Verizon had sent me a net promoter-style customer satisfaction survey about their Global Traveler Program (they didn't), I'd probably give it a 7. You can get an overview of Net Promoter scoring on the Net Promoter System website, but a 7 is a relatively neutral score.

It may be tempting to classify my service experience as good, but not great. However, my slightly positive overall perception of Verizon's Global Traveler Program is really a rough average of elements that were truly fantastic as well as some frustrating service failures that greatly diminished my enthusiasm. In other words, Verizon was one step away from either turning me into an enthusiastic promoter or an angry detractor.

What is Good Service?

Customer service is a function of each individual customer's expectations and experience. Outstanding service happens when experience exceeds expectations while poor service is the result of experience falling short of expectations.

What happens most often? Good service. This is when the experience meets expectations. There's nothing wrong with good service per se, but it isn't memorable. We only tend to notice, and remember, service that's outside the norm.

A Collection of Experiences

My perception of Verizon's Global Traveler Program was really a collection of experiences. There were at least seven distinct moments of truth that shaped my overall impression. 

My overall impression of Verizon's Global Traveler Program wasn't just impacted by the average of each moment of truth, but by their sequence.

Starting Perception. This signifies where a customer's perceptions are at the start of the experience. Good, or neutral, impressions are relatively easy to sway but strong perceptions are not. That's because a phenomenon called confirmation bias causes customers to selectively filter out information that doesn't match their existing beliefs. If they're a raving fan, they'll look for evidence that Verizon is awesome. If they hate Verizon, they'll look for any little nitpick to justify their feelings.

Primacy. My first two experiences with the Global Traveler Program were outstanding. Since my starting position was neutral, I was easily moved into a 9 or 10 positon on a 10-point scale. It's also important to note that first impressions are much more memorable than what happens next.

Mid-point. The two frustrating experiences were sandwiched between neutral or positive experiences. This, combined with an outstanding first impression, likely prevented my perception from being in the 4 or 5 range.

Recency. My last moment of truth was good since it was relatively easy to return the phone. Customers tend to remember their first impression and their last impression, so a good beginning and end can help overcome a few negative experiences in between.

What can we learn?

I see a few take-aways here:

1. Good service can hide distinct opportunities to be either great or terrible.
2. Companies should fall all over themselves to make a great first and last impression.
3. It's a good idea to collect data to help you spot the strong and weak points in your service delivery system.

What other lessons can companies learn from this experience?

---

Jeff Toister is the author of Service Failure: The Real Reasons Employees Struggle with Customer Service and What You Can Do About It. The book is scheduled to be released on November 1. You can learn more about the book at www.servicefailurebook.com or pre-order a copy on Amazon, Barnes & Noble, or Powell's Books.