Why don’t online reviews work as well as they are supposed to?

Why don’t online reviews work as well as they are supposed to?


The internet is primarily designed, and has evolved, to solve information problems. The internet cannot yet deliver experiences, except where the experience itself comprises entirely of information. The internet can only deliver information for the senses of hearing and seeing through pictures, text and video. Thus, any experience that comprises wholly of stimuli to these senses can be delivered, such as games. However, the internet cannot deliver information for the senses of touch, taste and smell. Thus, for us to experience stimuli to those senses, we must experience it in the ‘real’ world. All the internet can do is to deliver the information about those stimuli through text, and therefore, understanding. What it really means is that we receive an account of what the stimuli will comprise of, and in our minds we try to experience it. This exactly what online reviews are, and why they are becoming increasingly popular.

Everyday I come across different websites with a 5-star rating for reviews of something or the other. Even the most popular sites with reviews (yelp, youtube, etc.) do not provide a whole lot of value from the reviews, as more and more people add reviews. I am not saying that reviews are completely meaningless, but only that they do not completely encapsulate the information that they are supposed to.

However, the value of the reviews is measured by how accurately we feel those sensations that we expect to feel, when we actually do go and have that experience. Any time there is a gap between the expectation that we form in our mind versus the experience we have, the value of that information becomes suspect, and the source of that information gets discredited. In this context, the value of information can really be measured by this experience – expectation gap. It is worthwhile to note here that if that gap is positive (meaning that we end up having a better experience than expected), we are pleasantly surprised, while a negative gap induces disappointment.

Of course, this brings us to another problem, which is how to measure this difference between expectation and experience. Theoretically, the experience is captured in the description of the review, thereby contributing to the expectation. However, not all people are the same, and though the expectation from the same piece of information might be different for different people, what is more troubling is that the experience of different people varies a lot as well. An inherent assumption in the review model is that all reviewers are equal and that the set of reviewers is large enough to statistically represent the vagaries of human nature accurately. Thus, each reviewer gets an equal amount of voting power, while votes get averaged over many reviewers.

The upshot of this is that even though the amount of information contained in different reviews is different, it gets averaged over reviews to provide a more or less consistent amount of average information, which is enough to form a sort of personalized average expectation in the mind. For the present state of the internet, this is considered a fair system mostly because of a lack of a better automated and scalable system. This is the reason, however, why the average expectation for most “average” items (be it restaurants for yelp, or videos for youtube) tend to converge to ~3.5-4. The outliers are the superb (4.5-5) and the horrible (<3 stars). For the express purpose of classifying the item in one of these 3 buckets, the current review system is fine. But there is no real benefit from having the granular system of 5 stars, as the discrepancy between reviews is great. That is the reason we see a lot of websites today switching to the easier and simpler vote up/down system.

However, the truth still remains that the amount of information contained in each review is different, based upon the prior experiences and nature of the person generating the information itself. If we are able to capture this difference in a meaningful way, then the amount of information contained in a single unit of transport mechanism will greatly increase.
Facebook Comments
Comments are closed.
Get Adobe Flash playerPlugin by wpburn.com wordpress themes