Net Promoter Score Vs. Hively (pt. 1)
Posted on November 1st, 2011
One of the questions we get asked often is “how does the Net Promoter Score (NPS) compare to Hively?” Or, “is the Hively rating method similar to the Net Promoter Score (NPS)?” Our answer is that yes, it is similar. However, we believe that Hively improves upon the NPS rating system in a few ways.
Here are some ways we think the NPS method can improve:
1) NPS is not specific enough. The NPS method asks one question “Would you recommend our company?” While the question is great (and we’re all about asking only 1 question), from our perspective, it’s more important to get feedback on the specific transactions your customers have with your company, as opposed to getting general feedback on your company as a whole. People don’t interact with ‘companies’ as a whole. However, they do interact with individual products, services and people. Getting specific feedback gives you a more accurate measure of your overall customer happiness.
For example, how does the NPS score really help Apple? Apple makes amazing products that people love. So, when asked as a whole “Would you recommend our company?” most people would respond yes. However, if you asked an individual what they thought about the laptop they just bought that had the hard drive crash after 30 days, they might give you a very different and specific answer. Further, if a customer gives a negative rating on the new laptop s/he just bought, it doesn’t necessarily effect their overall opinion of Apple as a company.
When questions are focused and more transactional based the responses are much more useful, predictive and actionable to Apple as a company.
2) The Net Promoter Score has no value on it’s own. Meaning, the aggregated score only really matters in its relative comparison to other, similar companies in your industry. So, without the comparative measure, it’s difficult to assess what a good NPS score truly is.
3) The score itself is not enough. Why? Because there are multiple ways to get the same score. In terms of comparisons of companies within industries, NPS is problematic because there are many possible ways to arrive at the same score. For example, you can arrive at an NPS of 20 in dozens of ways. A company with a 20 NPS could have 20 percent promoters, 80 percent passives, and 0 percent detractors, while another company with a 20 NPS could have 60, 0, and 40, respectively. You could easily argue that the company with 20 percent promoters and 0 percent detractors is very different than the one with a polarized customer base with 60 percent promoters versus 40 percent detractors. Yet, their NPS score is the same.
4) The NPS scale needs labels. Most people would look at a 6 on a 10 point scale and think that’s 1 point above average – or slightly better thank ‘ok.’ However, a 6 in the NPS method represents a detractor – a negative result. But how many respondents know that? How would knowing change their rating? Labeling scales is imperative. In order to eliminate response bias and ensure that respondents are interpreting the question in the same way, the respondents must have a scale and all understand it in the same way.
5) The NPS scale is arbitrary. Ok, most scales are – but the 1 – 10 scoring system has seemingly become ingrained in people as the only way to measure satisfaction. That’s not the case. The NPS has an 11 point scale (0 through 10) where 0 through 6 are detractors, 7 through 8 are passive, and 9 through 10 are promoters. NPS is ‘unipolar’ in the way the primary question is phrased “How likely are you to recommend…?” while bipolar in how it’s applied (detractors versus promoters). A rating of 0 through 6 means “not likely at all to recommend,” which is very different than someone actually detracting or stating negative attributes and why. The application of NPS is inconsistent with the scale used, and would actually work better on a different scale like -3, -2, -1, 0, +1, +2, +3, which indicates “highly likely to recommend against,” through “neutral,” to “highly likely to recommend.”
6) The NPS method is biased and subjective. Statistically speaking, the NPS method is extremely biased toward detractors. Among the 3 categories, detractors have the widest range and therefore the highest probability (0 – 6). Meanwhile, passives and promoters each have only a range of 2 (7-8 and 9-10, respectively). When providing answer choices to a question or survey it’s best to have equal ranges and to have an allowance for a neutral response. This ensures the answers have equal weight and the respondents have an equal chance of picking each point of view represented.
Many NPS proponents might argue that the bias is by design, but this presents even more issues. Different societies and cultures have different ways of evaluating quantitative metrics and giving feedback. Some are much more guarded with their scoring, being much less likely to give higher ratings. Because of this difference and subjectivity, someone who gives a score of 8 might feel they’ve given you an amazing score. However, the NPS will categorize them as passive.
7) Perhaps most importantly, NPS is not actionable. What if you have a good NPS score? What does that mean? Say, for example, in one industry, ACME, Inc. has the best NPS. What does that really mean? Is it the best because of the technology, the service, the products? AMCE, Inc. is the company; but what product or service is actually worth recommending, if any at all? Without any real way of knowing it’s as useful as a Facebook Like button.
The opposite is also true. What if you had a bad NPS? What do you do? Is it a particular underperforming or unprofitable product or service that should be discontinued? Is it because you need to clean house on your customer support team? Without specifics, you have no way of knowing.
8) Transactional NPS still fails. Many NPS proponents have evolved their message (based on the shortcomings above) to suggest NPS can be ‘transactional’ based (as opposed to ‘relationship’ based). However, making this work correctly starts chipping away at the core of the NPS starting with “the ultimate question.” The ultimate question is supposed to be “Would you recommend…?” Again, asking this question of a company makes sense – but does it really when you ask it of a person? If a customer service rep asks you “would you recommend MY service?” what does that mean? It’s not like I’m going to tell my friends on Friday night about the amazing support rep I talked to this week. You could of course just change the question – but then you’re not really asking ‘The ultimate question’ as NPS has coined it – you’re just asking a short survey question. Because of this, all the failings above still apply plus the following two.
9) The NPS survey method is time consuming. Setting up an effective NPS process requires planning your questions, educating your team, distributing your questions, collecting responses, analyzing responses and then, lastly – acting on them. There is too much work to be done and too many gateways blocking the customer feedback from those who need it most – the customer facing team members.
10) NPS is rarely implemented properly. Mostly because of #9. Creating an NPS survey can be time consuming. Moreover, the data is typically aggregated and far removed from the customer facing teams. Companies therefore fail to properly analyze the results and worse – fail to take action on them or distribute the feedback throughout their organization.
In the next post in this series we’ll talk about how Hively is similar to NPS but greatly improves on this method. In recent months we’ve talked to several companies that have confirmed many of these issues with a NPS rating method. If you’ve had a similar or the opposite experience let us know in the comments!
View comments on this article
[…] system widely used to measure customer happiness. In the past, we blogged about why we believed Hively improved upon the NPS feedback method. However, the NPS method has recently been revised with a greater focus on the ‘system’ […]
[…] of the fundamental components of Hively are based on NPS. However, Hively improves upon the NPS rating system in several ways. Below we’ve listed all the ways we see Hively being the same and different […]