Customer ratings: likes, thumbs, or stars?

big picture ideas,customer experience | Friday, July 15th, 2011

Rating systems are an increasingly common feature on sites that catalog products or services. What may at first glance seem like a simple question – should the rating scale be thumbs or stars – actually reveals much more nuance on closer examination. Different rating scales are geared toward different contexts, and I think it’s important to understand what they communicate (and what they don’t).

This article addresses three types of rating schemes: one-dimensional (i.e. “Like”), two-dimensional (i.e. thumb up vs. thumb down), and multi-dimensional (i.e. the ubiquitous 5-star scale).

One-dimensional feedback

The simplest way to collect customer opinion is a single-click affirmation. Examples include:

  • Facebook’s “Like” button
  • A thumbs-up button by itself
  • Google’s new “+1″ button
  • Heart / favorite button (Kaboodle, Etsy, Slideshare, etc.)
  • Marking a comment as “helpful”
  • Flag as inappropriate

Kaboodle:

Vimeo:

This rating method is the simplest for the customer – it requires just a click, and usually doesn’t require going to a separate page. Frequently the “Like” or “heart” button is a toggle (on or off), and doesn’t ask the customer for written feedback. In essence, it allows a person to express some level of affirmation toward a product/content while investing very little time.

However, simple input also means simple output. There isn’t much you can do with the data, other than report back to the viewer how many people clicked the button. In the case of the Facebook Like button, the number of “likes” may be tied to your social network (“38 of your friends liked this”) which gives it a little more value. Other than social credibility, though, what does it really mean in the context of a product, or even a piece of content like this article?

“38 people faved this” means nothing if you don’t know how many people read it, or how many people totally hated it. Single-dimension feedback doesn’t give us insight into the range of opinions about something — it’s all or nothing.

The speed of the one-dimensional feedback mechanism makes it most useful for small content items, such as blog posts or videos. It would have little value applied to a complex product or service, where the nuances of opinion are much more important.

Note: there is much room for discussion about what it really means to “like” something in the Facebook era — it’s a shallow commitment at best. Don’t miss Jonathan Franzen’s NYT article “Liking is for Cowards: Go for what hurts.

With so many services offering ways to like, heart or share something, it’s getting a little ridiculous. That’s a lot of love to dish out over a simple blog post (Mashable.com):

Two-dimensional feedback

The next level of feedback allows a customer to submit an opinion on two levels: love it or hate it. The most ubiquitous example of this is the thumbs up / thumbs down scale.

YouTube uses a combination, pairing “Like” with a “Thumbs Down” button.

Again, this feedback mechanism doesn’t allow for nuances of opinion. It’s purely black and white; it omits the grey area, as well as the customers who didn’t vote one way or another, but might still have a strong opinion.

As with the single-dimension rating, this type of feedback is best for simple pieces of content (videos, forum comments, etc.).

Multi-dimensional: 5 stars and more

The multidimensional rating category is most often represented with the 5-star rating scale, which you now see everywhere, especially on e-commerce sites.

Amazon.com:

(Note: Amazon now combines the 5-star rating scale with a Like button: I guess it’s too hard to resist offering people a one-click “like” sentiment, as opposed to asking for a full written review, which requires both thought and time).

The 5-star rating scale has become the de facto standard for product reviews, in part because it’s automatically included with customer review platforms like Bazaar Voice and Power Reviews.

The advantage of a numeric scale (which doesn’t have to be 5 stars, but more on that later) is that it lends itself to better data. Customer feedback can be reported back as an average across ratings, and a small bar graph can also quickly show the distribution of responses (how much agreement there was among customer reviewers).

For a more complex product, many sites also allow customers to provide a rating of 1-5 on several attributes of the product. For example, AppExchange allows customers to rate a solution based on Ease of Use, Value and Support:

This highest level of detail obviously also requires the most time and contemplation on the part of the customer, so only the most committed fans (or the truly irate) will invest the time to complete such a review. The value of the data is much greater, but the trade-off is that far fewer customers will contribute, so the weight of each review could be disproportionate.

Finer points on rating scales

Most product reviews use the 5-point rating scale, but not all sites explain clearly what each point on the scale means. Responses will be more meaningful if each value is explained for the respondent — for example, “4″ means “I liked it, but didn’t love it” — rather than letting each respondent decide this for themselves.

There’s also room for some creativity here. Why use generic descriptors (“average”, “above average”) when you can customize the scale to match the product type or your audience? A site that sells to teenagers should have a different set of anchors (1= “this blows”, 5 = “OMG I’m in love”) than a site that sells washing machines. (Note to self: I don’t speak teenager very well. Need to brush up.)

Final note: the 5-point scale allows the reviewer to submit a neutral vote of 3, which may not always be helpful. If you want to force your reviewers to make a choice between the positive and negative ends of the scale, a 4-point “forced choice” scale is also a possibility.

Written reviews: the qualitative angle

Ratings & reviews together provide the most complete picture of customer interest. Written reviews provide the background to the simple quantitative measure of thumbs or stars, and are much more valuable to potential buyers of a product than a simple metric like “38 likes”. Of course, a well-written review is much harder to get compared to simple ratings (I won’t get into the topic of incentives here, but there are many creative ways to get customers to submit good reviews).

Some sites take the ratings & reviews form even further, and ask for information about the reviewer (Diapers.com: view full-size review; view full-size review form):

… or for specific attributes of the product like sizing, fit, or style (Bluefly asks for tons of information; see the complete review form here):

Asking for this much detail creates a review form that’s daunting to say the least! I’d be interested to know what the completion rate is for these types of reviews.

In conclusion

In conclusion, when you choose a rating & review system for your site, it pays to carefully consider what kind of feedback you really need. (Do you need to offer your readers 27 different ways to like something? Do you need to know your reviewers’ hair color or toothpaste preference?)

  • For a “that’s cool” affirmation of a simple piece of content, go with “Like” or “thumbs up”.
  • If you want useful quantitative information like average score or distribution of votes, go for a Likert scale (5 points to include a neutral choice, 4 points to force a choice), along with written reviews.
  • If your site sells a complex product or service, go ahead and add multiple product attributes, tags, and even reviewer demographics — as long as you’re confident your customers won’t be turned off by the length of the form.

I hope this was helpful, and please feel free to add your own experiences or opinions below. Cheers!

1 Comment »

  1. This Quora thread offers a really useful summary of research findings and resources for further reading on rating systems:

    http://www.quora.com/Is-there-a-better-alternative-to-the-5-star-rating-system

    Comment by Caitlin Gannon — November 22, 2011 @ 8:36 pm

RSS feed for comments on this post. TrackBack URI

Leave a comment

Powered by WordPress | Based on a lovely theme by Roy Tanck | | Log in