Imagine an inexpensive product which is functional for most people, but not full-featured. Many people who buy it think it works great and is a great value. Most people who buy it see its limitations, but appreciate the value. Some people feel the value isn’t so great. Some people have higher standards or expectations and don’t find the product worthwhile. A few people had major issues with the product, such as receiving a defective model.

This is a fairly typical situation, and we might naively expect the distribution of ratings out of five to resemble a bell curve shape centered around an average value of about four stars. However, product ratings aren’t usually distributed like this.

Instead, product ratings like this are usually distributed bimodally, with one peak at five stars and a smaller peak at one star, yielding the same average around four stars.

There are many factors that might explain this phenomenon. Here are a couple of my ideas.

Point 1: When a measure becomes a target, it ceases to be a good measure

This is an adage of statistics. It happens because people will try to affect the measurement itself in a specific way rather than allow it to function accurately. For example, a high-stakes exam will induce many people to attempt to cheat, because making the measure hit the target (getting a passing grade) is more important to them than having the exam accurately assess their knowledge.

In general, online reviews have no specific target. However, they often do for individuals. There are many people who think their opinion is more important or meaningful than others’. They don’t simply want their score recorded, they want the aggregate score to reflect their opinion. For example, someone who had a mildly positive experience at an establishment with very negative reviews might overrate in an attempt to raise the overall rating. This is especially true when the rating is emotionally charged or the rater perceives other reviewers to be incorrect.

Since this type of behavior is attempting to raise or lower the overall rating, it manifests in extreme ratings (i.e., 1/5 or 5/5). This means that fewer people select middle ratings, and the rating distribution is split.

Point 2: Statistical limitations of using a finite scale

There is another factor that may result in a bimodal distribution. Imagine a rating system where you can choose any integer rating, positive, negative, or zero. There is no maximum or minimum rating.

While this wouldn’t make sense for a number of reasons, it would hypothetically allow people to express their opinion of the product more accurately. If these integer ratings represent the “true opinions,” then the way they’re mapped onto 1-5 star ratings compresses everything at the extremes into the 1 and 5 bins.

So, for example, a person who thinks a product is simply not worth buying at all and a person who thinks a product is actively destroying the world both rate it 1 star, even though their preferences about the product are actually very different. (This is complicated by the fact that specific star ratings mean different things to different people.)

Instead of a distribution with long tails extending to the left and right, all the data is made to fit between two finite values. The tails aren’t “cut off,” but rather “bunched up.” In other words, even if the hypothetical integer-valued data would be distributed unimodally, it may become bimodal as a result of the finite restriction.

To be clear, the integer-valued ratings getting bunched up is just an analogy. What it corresponds to is how people’s actual preferences get mapped onto a 1-5 star rating scale. In particular, the idea here is that the two extremes represent more different preferences than the other bins.

Conclusion

In both points above, people select 1 star or 5 stars more often then other ratings because the rating system is collapsing into a binary choice of “I liked it” or “I didn’t like it.” In my opinion, this general effect is the most likely cause of the bimodal distribution. However, these are all hypotheses, and I don’t have direct evidence for any particular cause.

Do you find yourself rating things 1 star or 5 stars more often than other ratings? If so, why do you think that is?