Are 5 star rating systems useless?

I posted this in a thread in the Sword and Laser Goodreads forums, but I was curious what WordPress people thought about it.

Have you ever noticed that when you check the rating for a book on Goodreads, it will almost always be between 3.25 and 4.5 stars. Now, there are many factors that would account for this. 1 and 2 star books get abandoned because of their poor quality, and people don’t bother rating them or adding them to their shelves. (It’s true. The worst thing you can do to an author is ignore them.)

Some people are generous with their ranking because they don’t want to seem overly critical or hurt the author’s feelings, even if they’re not aware they’re doing it. Additionally, tastes differ and what may be the “best novel ever written” for one person is someone else’s “meh”. Even leaders in their genre like Tolkien and GRRM rarely rise above a 4.5. This is what happens when you try to apply the Law of Large Numbers to personal taste.

I thought it was strange, so I did a little experiment. I searched for 100 random common words and took 1 book (with >100 raters) from each list. From this I determined that the average rating for books on Goodreads is actually around 3.9 stars instead of the 3.0 you would expect. I repeated this with other lists, both random and selected, and they all came to similar numbers. (Mean~3.9, stdev~0.30. Try it yourself)

What does this mean? It turns out that about 70% of all titles are rated between 3.6 and 4.2 stars, while 99% fall between 3.0 and 4.8 stars. If 3 stars is average, about 99% of books are above average. So, should we adjust the scale so that 3 becomes a 1 and 4.5 becomes 5? While this might be more useful, it is practically impossible. Also, it makes fair to good books seem less appealing that they might actually be. The system is broken.

What can you do?

  • Rate all your books, especially the lousy ones. And be honest. You’re not doing anyone any favors by being nice all the time.
  • Double it. It may seem a little silly and even counter-intuitive, but it’s easy and helps put a rating in perspective by watering down the bias. Thus a 3 becomes an 6, and a 4.5 becomes a 9.
  • Read reviews. Notice the plural. Don’t base your choices on one person’s experience, even if it’s someone you trust. Look for a consensus opinion.
  • Write reviews. When someone takes the time to sit down and write a well thought out commentary, it says a lot about the book. Titles with few or no reviews are to be avoided.
  • Encourage Goodreads and others to adopt a more representative metric of a book’s quality than a simple average of votes.

What do you think? Should we give up on star ratings altogether?

Advertisements

Crazed recluse and sociophobe who has taken up writing after failing at everything else. Send pizza.

Tagged with: , , , , , ,
Posted in Blog, Things I know less about than I think
2 comments on “Are 5 star rating systems useless?
  1. Stars are pointless. IGN has it right with the point system.

    Like

  2. I think the star rating system is flawed, like you said. And you’re right — it’s because people don’t want to give a book a one or two star review unless it’s legitimately terrible. If it’s just meh, the book gets a 3, or even a 4. I think what we need to do is completely revise the entire rating system — maybe get rid of stars, and go with percentages … or even do like on some websites, where the buttons you can click aren’t stars, but say things like “I didn’t like it” “Meh” Pretty good” etc.

    Like

Comments are closed.

Now on sale

Unremarkable & Other Stories cover art

Sean Sandulak Unremarkable & Other Stories

Enter your email address to follow this blog and receive notifications of new posts by email.

Things I Will Probably Regret Later
June 2014
S M T W T F S
« May   Jul »
1234567
891011121314
15161718192021
22232425262728
2930  
  • I know it's a drop in the bucket for him, but Zuck's keeping the money from those Russian ads, isn't he? 6 hours ago
  • Telling yourself you're only going to play video games for an hour is the same as telling yourself you're only going to eat one potato chip. 1 day ago
  • Pfft. I bet @realDonaldTrump couldn't even find #Nambia on a map. 1 day ago
  • I, personally, am waiting for the gritty reboot of Mr. Roger's Neighborhood. https://t.co/hA3dAkqZIQ 1 day ago
  • Radio had a test of the emergency broadcast system. Considering the state of the world I decided to hide in the bathroom till it feels safe. 1 day ago
  • 1/I woke up with a headache and nausea this morning, so naturally I assumed that I had a brain tumor. (Hypochondria is a lifestyle.) 2 days ago
  • RT @Arikuyo: rt if you want to turn gaming into a gay multicultural jerk off session twitter.com/heterovet/stat… 3 days ago
Goodreads
Time until the end of the world
The Big DayApril 13th, 2036
18.6 years to go.