The Problem with 5 Star Rating Methods
Posted on November 29th, 2011
There are all types of rating methods online today. One of the most popular is the 5 star rating method. You’ve seen it everywhere from Yelp, to iTunes, to Amazon, to Netflix. Unfortunately 5 star rating methods face many challenges in their use, and numerous studies have identified flaws with this particular methodology.
The graph below illustrates the main issue with the 5 star rating scale. Most of the ratings are distributed to the 1 and 5 star ratings. Most people will only comment if they really like or don’t like something. The additional options of 2, 3, and 4 stars are unnecessary. After learning this, YouTube moved to a thumbs up / thumbs down rating system. However, taking all of the options out might be a bit much. Having a third, middle (the ‘meh’ preference) option in there is good to balance things out.
The bigger issue with 5 star rating systems is that in almost every case the ratings get averaged. When you average 5 star ratings you get two big problems.
The higher the average, the higher the item is listed in the scale.
Product A has 10 ratings, 9 of which are 5 stars and 1 is 1 star. Average rating of 4.6 stars.
Product B has 100 ratings, 85 of which are 5 stars and 15 of which are 1 star. Average rating of 4.4 stars.
Product A will be listed above Product B because it has a higher average. However, Product A has significantly fewer ratings and doesn’t necessarily deserve to be ranked above Product B.
Averages can be misleading and distract you from important details within specific ratings. Sometimes, just 1 review matters. We can’t find a better way to illustrate this point than with this comic from XKCD. This is a comic pretending to display ratings for a ‘Tornado Guard’ app that will alert you when tornado’s are near. The app has an average rating of 4 starts, but really, it’s only the last review that really matters.
Often times we get asked if the Hively, 3 option rating method (Happy, Satisfied, Unhappy) can be replaced with a 5 star rating system. The reasons above are just a few of the reasons why we think the 3 rating options within Hively are better. The simple smiley faces are universal, easy to understand and keep you focused on each individual rating as they come in – not an average. Give Hively a try yourself. It’s free to sign up and easy to get started. Questions about Hively? Let us know.
View comments on this article