Over the holidays Seth Godin decried some lazy analysis in a NYT piece which used the proportion of positive and negative reviews on Amazon to attempt to make the case that Amazon was failing its customers with the latest version of the Kindle. I'm not a Kindle fan, but he listed some good reasons for questioning the conclusions of the piece including this lovely little insight:
"Amazon reviews never reflect the product, they reflect the passion
people have for the product. As Jeff Bezos has pointed out again and
again, most great products get 5 star and 1 star reviews. That makes
sense… why would you be passionate enough about something that's sort
of 'meh' to bother writing a three star review?"
I'd like to add another to the list – the fact that it is almost without exception a small proportion of users on a site who account for the largest proportion of reviews or ratings, meaning the overall picture can be easily distorted and seem overly positive or negative. As proved by some research published late last year that studied the pattern of user-generated reviews across several sites including Amazon, IMDb and Bookcrossing. Standout ideas always generate polarity of opinion. But given our human tendencies toward bias, to see patterns where none exist, to do what other humans do, this is a timely reminder of the dangers of taking simple data at face value.