I like Amazon a lot, and I shop there a lot. However, there is a problem with Amazon's review rankings.
On August 4, 2013, I was ranked as the 3,806th highest reviewer on Amazon.com.
On August 5, 2013, I was ranked as the 2,933rd highest reviewer on Amazon.com.
I don't know how many reviewers are on Amazon, but I saw somebody ranked in the 16,000,000 range recently, so this puts me in the 99.98th percentile of Amazon reviewers--the top 1/50th of one percent, which means a great many people have found my reviews helpful over the years.
This begs the question: How did I go from top 4,000 overnight to top 3,000?
Answer: By reviewing less; by sharing less of my opinion. We'll get back to that later.
Having posted my first review almost a decade ago, I had worked up to 261 reviews on August 4. My total "helpful" score was 72%; for every opinion on my review, just over seven said it was helpful. I reviewed on many things: gadgets, books, tools, movies, etc.
Perusing my review history, I noticed an undeniable trend: People thought I was pretty good at reviewing tech products; I received a high percentage of helpful votes. However, people thought I was pretty bad at reviewing movies and books; I received a low percentage of helpful votes.
Objectively, some of my low-helpful reviews were poor; maybe I wrote a sentence or two, gave the product a 1/5 and moved on. However, some of my reviews were objectively thorough, and I still received a low score. I noticed other reviews of the same products suffer the same fate.
I started to realize that the problem wasn't me: it was Amazon's review trolls.
The above is not a unique opinion. In this large study Amazon's top reviewers , we learn a great deal about reviewers' behavior, including this statement:
“a positive review of a conservative politics title is sure to attract a great number of “not helpful” votes by those who don’t like the author’s politics.”
and
Books that have a “fanatical following” are, however, avoided because any negative review of that item is likely to be voted “not helpful”.
An excellent example of review trolls had to do with a book on Michael Jackson.
Further:
Most reviewers avoid writing negative reviews because such reviews are more likely to be found “unhelpful”.
Amazon's review ranking algorithms rewarded me, a well-ranked reviewer, by reviewing even less. Therefore, either it's working well because it's encouraging me to stop reviewing certain subject matters for which I have little expertise and/or a poor ability to review them, or it's working unwell because it's encouraging me to avoid subject matters that would benefit from my insight. There is surely some of the former, but I believe a great deal of the later.
Many analysts also believe a lot of review on the net (including on Amazon) are simple fakes, put there by the product maker to promote their product, or put there by competition to denigrate another.
On August 4, 2013, I was ranked as the 3,806th highest reviewer on Amazon.com.
On August 5, 2013, I was ranked as the 2,933rd highest reviewer on Amazon.com.
I don't know how many reviewers are on Amazon, but I saw somebody ranked in the 16,000,000 range recently, so this puts me in the 99.98th percentile of Amazon reviewers--the top 1/50th of one percent, which means a great many people have found my reviews helpful over the years.
This begs the question: How did I go from top 4,000 overnight to top 3,000?
Answer: By reviewing less; by sharing less of my opinion. We'll get back to that later.
How Amazon Ranks Its Reviewers
Amazon has an unpublished set of logic it uses to rank its reviewers. Among other things, this logic looks at number of reviews a person has posted, how recent they are, how many "helpful"/"not helpful" votes have been received, etc. The formula behind this is not publicly available.Having posted my first review almost a decade ago, I had worked up to 261 reviews on August 4. My total "helpful" score was 72%; for every opinion on my review, just over seven said it was helpful. I reviewed on many things: gadgets, books, tools, movies, etc.
Perusing my review history, I noticed an undeniable trend: People thought I was pretty good at reviewing tech products; I received a high percentage of helpful votes. However, people thought I was pretty bad at reviewing movies and books; I received a low percentage of helpful votes.
Objectively, some of my low-helpful reviews were poor; maybe I wrote a sentence or two, gave the product a 1/5 and moved on. However, some of my reviews were objectively thorough, and I still received a low score. I noticed other reviews of the same products suffer the same fate.
I started to realize that the problem wasn't me: it was Amazon's review trolls.
Amazon's Review Trolls
Review trolls are those individuals who will mark a review as helpful or not helpful based only upon its rating score, regardless of its actual content or worthiness as a genuine and decent review. Review trolls tend to congeal around divisive topics, and products that entice emotion: books, movies, music. For example, if a reviewer gives a poor score to a cult classic film, many of its fans will mark the review as not helpful. Alternatively, if a high score is given to something that the vocal minority has determined is a poor product, positive reviews will have a difficult time acquiring helpful votes.The above is not a unique opinion. In this large study Amazon's top reviewers , we learn a great deal about reviewers' behavior, including this statement:
“a positive review of a conservative politics title is sure to attract a great number of “not helpful” votes by those who don’t like the author’s politics.”
and
Books that have a “fanatical following” are, however, avoided because any negative review of that item is likely to be voted “not helpful”.
An excellent example of review trolls had to do with a book on Michael Jackson.
Further:
Most reviewers avoid writing negative reviews because such reviews are more likely to be found “unhelpful”.
Improving Amazon's Review Ranking by Deleting Reviews
I decided to run an experiment. As mentioned, I had 261 reviews with 72% helpful score. I went through my entire list of reviews and deleted 30 of them. Virtually all of these were over books, movies, or music. I waited until the next morning and, with 231 reviews had an 84% helpful score, my ranking sky-rocketed.Amazon's review ranking algorithms rewarded me, a well-ranked reviewer, by reviewing even less. Therefore, either it's working well because it's encouraging me to stop reviewing certain subject matters for which I have little expertise and/or a poor ability to review them, or it's working unwell because it's encouraging me to avoid subject matters that would benefit from my insight. There is surely some of the former, but I believe a great deal of the later.
Why Would an Amazon Reviewer Game the System?
Motivations to have a high review ranking score are many, but for the top reviewers a high score can equal money. Top reviewers are often part of Amazon's Vine program, which means they can get products for free.Many analysts also believe a lot of review on the net (including on Amazon) are simple fakes, put there by the product maker to promote their product, or put there by competition to denigrate another.
How to Fix Amazon's Review Rankings
Some ideas. I'm sure there are many more!- Deleted reviews should still count for, or against a reviewer
- Vote-spamming should be addressed. This means if an individual marks as helpful or not helpful a series of reviews in very short-order (i.e. a couple of seconds apart), their votes can in some manner receive less, or no importance.
- Address the bias to early-reviews. The first thing a customer on Amazon sees is the "most helpful" reviews. These reviews see a lot of attention, receive more votes, and cement further their position as most helpful. Top reviewers know this and, according to the study of their behavior, many only review products with few, or no reviews, lest their review get lost in the noise.
Additional Reading on the Problems With Amazon's Review Rankings
- Can You Trust an Amazon Review?
- How Amazon Should Fix Its Reviews Problem
- Fake Reviews: Amazon's Rotten Core
No comments:
Post a Comment