Let's talk rating systems

This is something I've been thinking about recently, I've been considering writing a little about board games, and I'm curious what folks think of different rating systems (binary, 4 star, 5 star, 10, 100).

I understand the rating (grade) is the least important part of the review, but when I started researching how people used them (Ebert, Meltzer, Mark Johnson of bgtg), I found it somewhat divisive.

Currently I'm feeling the Siskel/Ebert thumb system might be best. While I see the merit in a numbered scale, the out of 5 and out of 10 systems both have numbers (3 and 7 respectively) that tend to collect more ratings than the extremes, while being the least helpful.

I mention Mark Johnson's (5 star) system because he describes it as avoiding 7's and encouraging 10's. I like the concept of committing a game in that way, thus avoiding half stars, and having to decide if a game is a 6 or an 8.

there has been a lot of discussion on number systems in the world of video game reviews. Tom Chick, RPS and others have taken various stances on the issue.

out of 100 allows for the most precision, but also the most meaninglessness. I don't think you can really say that one game is 67 and another 69 in any meaningful way.

Out of 10, and this is true of out of 100 too, has the problem that the majority of people don't use 4 and 5 that much. The majority of games are ranked between 6 and 9 with 1,2 and 3 reserved as a for particular scorn on games. Essentially most people end up using it as a 5 point scale. The question arises, why not use the whole scale? The problem is, if you decide that 6 is actually a good score and everyone else thinks its bad you end up miss-communicating, especially in the real of metacritic.

out of 5 gives you broad classifications, and i think is probably the best. It gives you a number that allows you to say broadly whether something is good or bad. Some people will feel it doesn't give them enough scope to differentiate between two games that the like slightly differently. My point is that i don't care about slight differences in opinion/quality as my view is not likely to be the same anyway.

Binary is ok. I do like to know with a rating whether something is good or exceptionally good though.

Why do you feel you need a rating? I find the language used by a writer more meaningful than a generic number that is interpreted by people in different ways.

charlest wrote: Why do you feel you need a rating? I find the language used by a writer more meaningful than a generic number that is interpreted by people in different ways.

In short, I don't feel I need one. When I was researching criticism this was something I found interesting and wanted to talk about to flesh out my thoughts on.

I really like the idea of a 5-tier rating system. I might switch mine to that.

I confess, I sort of love rating games on BGG. It's one of the most useful components of BGG for me, the ratings and comments. I frequently gauge the GeekBuddy response to games I'm looking into, so I like to make sure my own ratings are useful to those who use the same tool.

Having said that, I've gotten really sloppy with rating stuff because of exactly what you say. Is this a 6 or a 7? What's the difference? Increasing that granularity between ratings would probably make them a little more useful for me and for everyone.

DukeofChutney wrote: out of 5 gives you broad classifications, and i think is probably the best. It gives you a number that allows you to say broadly whether something is good or bad. Some people will feel it doesn't give them enough scope to differentiate between two games that the like slightly differently. My point is that i don't care about slight differences in opinion/quality as my view is not likely to be the same anyway.

Binary is ok. I do like to know with a rating whether something is good or exceptionally good though.

On the topic of 5 star vs. binary, 3 star is the weird spot.

I liked the 5 star concept so I switched all my BGG ratings to that, across the board, I would give a thumbs down to 1 and 2 rated games, and thumbs up to 4 and 5 games. In 3 there are games I would give both to and I'm not sure what's more helpful, a problematic middle vote, or a thumbs up/down (which is a decisive endorsement).

The use of a rating system is for folks to quickly glance and see how the reviewer felt about the game. Siskel's thought on the thumbs was it would express decisively "go see the movie" or "don't see the movie". 1,2,4,5 all do this, what does a 3/5 tell anyone though? I guess it tells them to read the article, but with the games I've rated 3 is the most common number.

I'm fine with the 1 to 10 on BGG but I'll be honest -- I use it more as a ranking tool than anything else. I'll give a game a 7.5 and look and see: oh my, this game isn't as good as Ticket to Ride Europe and I gave that one a 7.5, so let's back this off to 7.2. The result is that I have a lot of weird numbers in my ratings.

S.

I really need to go through all my ratings and revise them. I've become grumpier and more brutal in my old age and less patient with games that serve no purpose except to sit on a shelf. There are things I've rated a 7 that need to be moved down to a 5 because they seemingly aim right for a decidedly average experience.

I am guilty of pretty much only using a 6 point scale. 1-2 is garbage. 3-4 isn't garbage, but I see no reason to play it. 5-6 is average, run of the mill, Oh Look Another Cubes On a Ship game. 7 means I don't mind playing it, and I'll even ask for it sometimes, but not a real attention getter. 8 is the weirdo grade. It's usually assigned to things I've played once and found interesting. They often either get turned into 9s if more plays show it as a good game or 7s if more plays don't bear out my interest. 9-10 are more or less identical, though I save 10 for my super extreme power favorites with nitro express.

I'd prefer a 4 point scale. Something's either:
1) terrible
2) bad side of mediocre
3) good side of mediocre
4) great

And yes, then you're writing fills in the details and nuances more than a decimal place would.

All numerical/star rating systems are trash. The only one that works is the classic Siskel & Ebert thumbs up/thumbs down one. This is because either a work is reviewed favorably or it is not, all degrees of value, details and fine points should be supplied by the critic's language. Games should literally be rated either with a "yes" or "no" metric- zero or 100. If it's 100, it's recommended because of reasons A, B and C. If it's 0, it's not recommended because of reasons X, Y and Z. The end.

This shit ain't rocket science.

Michael Barnes wrote: All numerical/star rating systems are trash.

Depending on your perspective, sales numbers is a pretty solid indication of value. Generally that doesn't apply to end-users.

I'm more of a ranking guy. But even that is fleeting.

S.

Funny, I was just writing about this myself.

I think the main reason to use a rating system is to quickly tell the reader how the writer felt about a game and help them find articles they are interested in more easily.

For example, I wrote about an obscure game called A-Train 9 which was trashed by most mainstream reviewers and ignored by the majority of the populace. If I didn't attach it an score, nobody would care about it. "Just another game I kind of heard was terrible". They aren't even going to read my review and I don't blame them.

But I gave it a very high score, as high as the one I gave to Chicago Express. And I've noticed that many boardgamers who have played Chicago Express think "Chicago Express was great and this dude says A-Train 9 is as good as it, perhaps I should check it out" and read my A-Train review.

And it's also great for dissenting opinions. Most people won't read yet another Arkham Horror review, since they already know the game is good. But I don't think the game is good and gave it a negative review. Now, I have a dissenting opinion and there's value in that, perhaps it can help my readers understand the game better. Without a score attached, most people wouldn't noticed I have a dissenting opinion and would simply ignore the article.

It also comes handy for those times where just words don't quite convey how "better" or "worse" a game was compared to another. It is very difficult to explain why Super Jump Game gets 4 stars and Platformer King only gets 3 when the answers are "level design", "tightness" or "interesting enemy patterns" and often you don't even want to explain those differences too much because it would make the review totally unfocused. So having a score comes very handy.



The bad side is that many readers focus very heavily on that aspect, perhaps more than it would be desired. Some critics blame them for it and it's not like they don't deserve some blame but c'mon, you are the one explaining things, complaining that the reader "doesn't get it" doesn't seeem to productive to me.

Hence my recommendation is: If you want to use a rating system try not to ram that focus head-on because most people won't get it and it's more educative to try another approach. So..if you are going to avoid the "A seven means bad game" phenomenon don't use the 10 point or 100 point scale because you will only confuse your readers.



I use letters instead of stars right now because I don't know how to use Wordpress but this is the scale I want to use. It's basically a 5 star scale with a sixth "no star" rating so buggy and other completely awful games don't take a good chunk of the scale and put the "average" significantly above 2.5 Stars.


No Stars (20)- Extremely dull or barely working. There's often very little reason to play these games, much less write about them and so this rating goes mostly unused.

Example: LCR. You simply don't want it no matter how soft you are.

1 Star (38) - The game is more defined by its flaws than by its good side. These games often have poor ideas, a bad execution or just dull premises but they can be enjoyed with some work.

Example: Munchkin. It's not good but it's not like it's going to kill your parents. You may be happy to own it.

2 Stars (52) - Good games that don't quite make it. They are interesting, they are fun but they lack something important and can't quite stand to the competition.

Example. Eurocube 34. It works well, but it won't make you jump out of your seat. It may become a personal favourite.

3 Stars (68) - Three star games are games that stand out. They are well-made, with very interesting aspects and while they aren't perfect, they are more than worthwhile.

Example: Martin's Wallace Eurocube. It's like Eurocube but without the boring parts. Totally worth owning if you like it.

4 Stars (84) - A genre classic. The game will endure as a great example of what a good game is.

Example: Chicago Express. One of the best train games around. But it's not one of the best games in the market. (IMHO)

5 Stars (100) - Everything else.
Example: Steam. It's not just one of the best train games, it's one of the best boardgames you can get right now.

I still haven't written the definitions, I want them to sound very positive instead of "LOLZ THIS GAME SUCKS".

I chose a 5+1 star scale because it's the lowest number of categories I'm confortable with. My recommendation is to get a bunch of games you think are significantly different in quality from each other and see in how few categories you can split them.

If you have too few categories, there will be more cases where two games of very different quality have the same rating. If you have too many, you get the 7 point scale.

So yeah, that's what I think, I guess. Hope that helps.

I've always felt the three tier of:

Weak 7
Solid 7
Strong 7

Really tells me all that I need to know about a game.

If im reading an engaging review i don't expect a score. If i'm looking through what someone thinks of the games in their collection to try and gauge their interests, or if i want crowd consensus on whether people liked a game or not scores are helpful.