Blog, News & Announcements

The Problem with 5 Star Rating Methods

Posted on November 29th, 2011

There are all types of rating methods online today. One of the most popular is the 5 star rating method. You’ve seen it everywhere from Yelp, to iTunes, to Amazon, to Netflix. Unfortunately 5 star rating methods face many challenges in their use, and numerous studies have identified flaws with this particular methodology.

The graph below illustrates the main issue with the 5 star rating scale. Most of the ratings are distributed to the 1 and 5 star ratings. Most people will only comment if they really like or don’t like something. The additional options of 2, 3, and 4 stars are unnecessary. After learning this, YouTube moved to a thumbs up / thumbs down rating system. However, taking all of the options out might be a bit much. Having a third, middle (the ‘meh’ preference) option in there is good to balance things out.

The bigger issue with 5 star rating systems is that in almost every case the ratings get averaged. When you average 5 star ratings you get two big problems.

Problem 1:
The higher the average, the higher the item is listed in the scale.

For example:

Product A has 10 ratings, 9 of which are 5 stars and 1 is 1 star. Average rating of 4.6 stars.

Product B has 100 ratings, 85 of which are 5 stars and 15 of which are 1 star. Average rating of 4.4 stars.

Product A will be listed above Product B because it has a higher average. However, Product A has significantly fewer ratings and doesn’t necessarily deserve to be ranked above Product B.

Problem 2:
Averages can be misleading and distract you from important details within specific ratings. Sometimes, just 1 review matters. We can’t find a better way to illustrate this point than with this comic from XKCD. This is a comic pretending to display ratings for a ‘Tornado Guard’ app that will alert you when tornado’s are near. The app has an average rating of 4 starts, but really, it’s only the last review that really matters.

Often times we get asked if the Hively, 3 option rating method (Happy, Satisfied, Unhappy) can be replaced with a 5 star rating system. The reasons above are just a few of the reasons why we think the 3 rating options within Hively are better. The simple smiley faces are universal, easy to understand and keep you focused on each individual rating as they come in – not an average. Give Hively a try yourself. It’s free to sign up and easy to get started. Questions about Hively? Let us know. 



View comments on this article

Happy customers make happy teams

Free for teams of 3. No credit card required on signup.