Are 5 star rating systems useless?

I posted this in a thread in the Sword and Laser Goodreads forums, but I was curious what WordPress people thought about it.

Have you ever noticed that when you check the rating for a book on Goodreads, it will almost always be between 3.25 and 4.5 stars. Now, there are many factors that would account for this. 1 and 2 star books get abandoned because of their poor quality, and people don’t bother rating them or adding them to their shelves. (It’s true. The worst thing you can do to an author is ignore them.)

Some people are generous with their ranking because they don’t want to seem overly critical or hurt the author’s feelings, even if they’re not aware they’re doing it. Additionally, tastes differ and what may be the “best novel ever written” for one person is someone else’s “meh”. Even leaders in their genre like Tolkien and GRRM rarely rise above a 4.5. This is what happens when you try to apply the Law of Large Numbers to personal taste.

I thought it was strange, so I did a little experiment. I searched for 100 random common words and took 1 book (with >100 raters) from each list. From this I determined that the average rating for books on Goodreads is actually around 3.9 stars instead of the 3.0 you would expect. I repeated this with other lists, both random and selected, and they all came to similar numbers. (Mean~3.9, stdev~0.30. Try it yourself)

What does this mean? It turns out that about 70% of all titles are rated between 3.6 and 4.2 stars, while 99% fall between 3.0 and 4.8 stars. If 3 stars is average, about 99% of books are above average. So, should we adjust the scale so that 3 becomes a 1 and 4.5 becomes 5? While this might be more useful, it is practically impossible. Also, it makes fair to good books seem less appealing that they might actually be. The system is broken.

What can you do?

• Rate all your books, especially the lousy ones. And be honest. You’re not doing anyone any favors by being nice all the time.
• Double it. It may seem a little silly and even counter-intuitive, but it’s easy and helps put a rating in perspective by watering down the bias. Thus a 3 becomes an 6, and a 4.5 becomes a 9.
• Read reviews. Notice the plural. Don’t base your choices on one person’s experience, even if it’s someone you trust. Look for a consensus opinion.
• Write reviews. When someone takes the time to sit down and write a well thought out commentary, it says a lot about the book. Titles with few or no reviews are to be avoided.
• Encourage Goodreads and others to adopt a more representative metric of a book’s quality than a simple average of votes.

What do you think? Should we give up on star ratings altogether?

About Sean Sandulak

Crazed recluse and sociophobe who has taken up writing after failing at everything else. Send pizza.