On a scale of 0 to 10, how likely are we to recommend the Net Promoter Score to a friend or colleague? Guess… ☹
Musings on the limitations and drawbacks of the Net Promoter Score
What are the reasons for using the Net Promoter Score
What are the considerations you should take into account when using
What can you do instead to measure (and improve) customer experience
The Net Promoter Score (NPS) has become ever-present in the world of customer experience evaluation. It is the de-facto standard metric for measuring customer sentiment and many brands and business leaders feel it is the most meaningful indicator of customer-centricity that they need to focus on. It is not hard to understand why:
It has widespread acceptance in the business world. Having been in use since the early noughties. The wisdom of crowds dictates if others are doing it you should probably be doing it too.
As a widely used standard, it also means it is a metric that has a lot of benchmarks, if you want to understand how you are doing vs. your competitors and have enough money.
On the face of it, it is an intuitive metric based upon some clear categorisations or groupings which feel customer-centric, the opposition of promoters vs. detractors is a Manichean conflict for business execs to grapple with.
It is a composite metric that has a foundation of pseudo-science. It feels clever, you can read books about it. It has an element of sophistication over and above a single dimension metric such as satisfaction. But, equally, it is not a black box; it is knowable and breaks down your customers into differing cohorts based on how they feel about your brand.
For many businesses, it is not an option to not use NPS. The reality is though that it is bad research, the analysis misrepresents the data and, as a result, it does a poor job of helping you understand your customer and their relationship with you. So use it if you must but be mindful that there are better ways to measure how your customers feel about you and understand what you need to do to improve this. And these can be employed alongside NPS in lean CX measurement - it is not a zero-sum game!
Lots of research has been published to disprove the claim that NPS is a good predictor of future customer loyalty. So why are we sharing this piece? Mainly as we find it concerning the amount of NPS how-to guides we read — often from CX research providers which fail to flag up any of this criticism. Toeing the line is good for business clearly, but still it is quite shocking.
We also feel obliged to add our critical take to the dissenting voices as we think NPS is both a poor barometer of current CX and future customer loyalty but is also a poor diagnostic of how to improve customer relationships. And we would be poor advocates for CX research in general if we didn’t aim to help businesses optimise how they measure CX and act on the insight! We will follow this piece up with a companion piece to examine ‘If not NPS then what?’ to elaborate on some of the themes and thoughts outlined below.
One of our number recalls listening to an interview Satya Nadella gave to The Verge a few years ago about the future of Microsoft, (complete disclosure this was 2015). Towards the end of the interview Nadella was asked about future challenges and said…
“At Microsoft we no longer talk about the lagging indicators of success, which is revenue; profit… what are the leading indicators of success? Customer Love; there is actually a way of measuring it, NPS, Net Promoter Score.”
See for yourself here at around the 9.45 mark. A few things that leapt out at us:
1. The lagging indicator / leading indicator framing is super cool, and is something we have absorbed into our own vernacular since without credit (sorry Mr Nadella)
2. NPS gets a lot of traction, you see it on the first page of a lot of annual reports but to hear the head of a £2tn dollar business reference it makes this quite explicit. The journey of NPS has been from consultants, to boardroom to investors and now beyond.
3. NPS can be whatever you want it to be, recommendation, satisfaction, even customer love as Nadella puts it. Hey, it’s a number you can crow about when it goes up, and as good a reason as any to fire people when it goes down.
4. Ergo, and not surprisingly, not many people outside of consultants & insight teams really care what it is or how it works — they trust the experts.
5. And finally, looping back to the first point, yes NPS may be a ‘leading indicator’ but is it a good one let alone the best one?
In or view almost certainly not. So we are going to attempt to deconstruct this and explain why:
1.The Net Promoter Score does a poor job of help businesses understand what they need to do to be more customer centric.
The Net Promoter Score is a measure of customer recommendation, what it can’t do in isolation is help you understand what you need to do to improve customer loyalty or recommendation. It is often used in conjunction with an open-ended ‘drivers’ question — ‘why did you give this score’ to unpick why someone ‘promotes’ or detracts’. But even with the benefit of AI text analytics this is not the most incisive way of understanding what your brand, product or service needs to do to be more competitive.
2. Net Promoter Score classifications are misleading — do detractors really detract?
The Net Promoter Score lexicon is misleading; businesses lean into the language of promoters and detractors often without a thorough examination of whether people are in fact advocates or critics. The problem is that the method does not address this directly as part of the approach. People haven’t been asked directly whether they speak positively or negatively about a brand (if at all) however once the data starts getting reported and the labels are applied then this is how they are viewed. As a reminder this is what the respondent question looks like:
3. Partially coded scale = ambiguous data
So, looking at the example of the above, what exactly does providing a seven or a three or any other response between ten and zero mean? The research participants have not been asked if they go around singing your praises or slating you. And an ambiguous undefined scale leaves you exposed to cultural and other biases when interpreting survey data.
As a foil to this, you may sometimes see an incredibly leading implementation of the question along these lines:
Which prompts people to provide a 7+ unless they are unhappy with the brand, product or service (and the visual language unhelpfully has nothing to do with recommendation more sentiment and satisfaction). And begs the question why not just condense the scale to three points!
To identify ‘promoters’ or ‘detractors’ it really is better to ask the question directly.
When we do have to use the NPS metric we tend to combine with what we call ‘active recommendation’ or referral:
Much better, we are no longer dealing with hypotheticals. We can understand are you being talked about and is the sentiment positive or negative. People answering the question understand how their answers are being categorised. Unsurprisingly in all programmes where we have paired NPS with active recommendation, the former was a poor predictor of the latter.
4. The theory underpinning NPS posits that customer loyalty / advocacy is the key metric all businesses should be focused on.
Is this true and does this reflect human behaviour?
Some examples:
· Many people love the BBC, does this mean they go around recommending BBC1 to people? Possibly not, depending on context, it could come across as slightly odd.
· You may be super satisfied with your energy supplier — is this something you can chat to your friends about down the pub without provoking excuses for an early departure?
· You are delighted with how that topical ointment you got from the chemist cleared up that breakout of….. sorry no need to go on.
· These examples show that brand or product satisfaction need have no bearing on likelihood to recommend. If recommendation is contingent upon other factors other than satisfaction and the conditions for recommendation can vary from category to category — why get bogged down with it in the first place?
5. A composite metric obfuscates
Here’s a great example of how NPS can obscure meaning:
Two brands A and B both have an NPS of 20. So even stevens? Perhaps not… Brand A in the example below has no negative response (detractors) but a weak positive response (promoters), a bit meh! Brand B feels altogether more polarising, with high levels of detractors and promoters. We are not going to posit which is a better place to be, only that the NPS composite metric obscures the difference if reported as a single score. Not a great way to help understand your brand or customers!
6. The volatility of the metric obscures meaning
The below, is a hypothetical example of one of many customer satisfaction trackers we have run was included. The monthly sample size is about as big as it could be based on having at least a million engaged customers:
A relatively stable satisfaction metric vs. an NPS score which is all over the place. As it’s a composite metric there is increased volatility and noise, therefore you either need larger sample sizes or a longer horizon to view the data over. The constant up and down of the NPS score vs. a more stable satisfaction metric led in most cases to CX autopsies to explain why it was shooting up and falling back when in reality these changes from month to month were not meaningful. In short, another example of why the composite aspect of NPS is a weakness; not a strength.
We will conclude here for now. And we are in no doubt that regular users of NPS may have independently arrived at similar thoughts to those expressed above. And this is by no mean a comprehensive list of flaws and failings.
Whilst NPS is not going anywhere soon, we would implore users to consider the limitations of NPS. Not only when analysing or reporting the data but also when designing the research. If your focus is really on improving CX, recommendation, loyalty and acquisition, there are better ways to both measure performance and provide a diagnosis that will lead to brand growth in every sense of the word.
We will follow up soon with an outline of what this could look like and will leave you with the thought… forget Net Promoter; think Need Fulfilment.
Customer Experience Surveys
