Sections

Why review scores are dead | Opinion

TinyBuild CEO Alex Nichiporchik explores why critics' ratings are not always indicative of a game's true quality

Review scores used to be able to tell you if a game was good. Magazines and sites often broke things down into individual scores aspects, and if the story scored a 9/10 and you liked story-driven games, it was a good indicator that you should probably buy the game.

This was before you were able to watch someone play the first half-hour of the game and before you could see the community react. Shelf space was limited, and there was no Twitch or YouTube.

Gamers make their own decisions now. And quite often, that decision does not align with the opinion of an editor that's giving a review score.

Here are the reasons why review scores are dead and what developers should do about it.

Games are becoming experiences

When we launched Hello Neighbor, a major gaming outlet gave it 4 out of 10. Granted, there were bugs, and we spent a lot of time fixing them. But it's also important to note that reviewing something like Hello Neighbor is kind of like reviewing Fortnite. What is the final product? Is it the current season? The rap concert? No, these are experiences within the bigger scope of the game.

head

Alex Nichiporchik, TinyBuild

With Hello Neighbor, this experience started when we released our first playable alpha. One time, we accidentally forgot to disable the dev console. It opened up the backend to the community and allowed them to see what was being worked on, what was a work in progress. It became an experience on its own -- seeing the behind the scenes, the models, and how the game's final form was shaping up. Playing the game in its development form became part of the game itself.

Younger gamers don't care about reviews as much

It's important to distinguish between different generations of gamers. People over 30, who, like myself, grew up using scores are limited on time. So review scores will factor in for that specific audience.

However, the younger generations of gamers, who grew up online, with phones without buttons, are right now the biggest gaming community in the world.

When did this shift from finished product to experiences happen? I think it was the first generation of Minecraft gamers. These were 12 year olds during the early 2010s. They are the ones that started playing a lot of emergent games that were not published and marketed by traditional means.

Show yourself early and win

To truly succeed today, you need to start showing your game early. It's not about avoiding review scores. The goal is to help your potential customers see if they actually want to play the game.

Once you start building up your community, they will be the ones that react and actively engage on different media websites. They will play demos and betas, become active on Discord and drive the popularity of your game.

You really just need one thing: to be honest and transparent. Because it's not about the product itself. It's about the experience and the community that the game creates.

Games are much less static

While games have always seen patches and updates (even the earliest arcade games got revisions, and DOS games spread patches via magazine cover disks), they pale in comparison to the transformative updates games frequently receive now.

Reviewing something like Hello Neighbor is kind of like reviewing Fortnite. What is the final product?

Even outside of early access, there's a huge number of titles that have seen major overhauls or content updates that would change a review score, but there's almost no outlets that re-review games. That original score, however misrepresentative it is of the game that exists now, is what people see when they turn to Metacritic.

The problem is even more apparent for live service games. You have only to look at the Metacritic scores for Warframe, mostly from reviews written in 2013, describing a game completely different to the one that exists now. While an interesting read, the score is what people often see first, and if it's misrepresenting the game today, it helps nobody.

In TinyBuild's case, our own Hello Neighbor wasn't particularly well received by the press when it first released, and I can't deny that part of that was due to bugs and other technical issues. But with a little spit, polish and elbow grease, we're proud of the game as it stands now.

Given Hello Neighbor's Very Positive user rating on Steam, we think our audience agrees, but if you were to look at those initial reviews, you'd think that nothing had improved.

User Reviews can offer more up-to-date info

While every system has its problems, Steam having seperate scores for recent and lifetime user reviews, as well as filtering out review-bombing, can tell people a lot at a glance. You have only to look at No Man's Sky to see the benefits of this system, with the overall score (hurt by a rough launch) overshadowed by glowing reviews of the current version of the game.

At TinyBuild we've also observed that if an initial PC release has a high user rating on Steam, then it's likely to sell well when a later console release happens.

Given how unreliable old review scores can be, we believe that people are looking at more recent reactions on Steam, YouTube and other outlets to help decide what to purchase.

Of course, this isn't to say that user scores are without issue. Review-bombing on Metacritic is rife, and Steam had to implement systems to deal with this problem on their own turf. Fixes that few other sites have made.

Games like Hello Neighbor change over time, but their review scores do not

Games like Hello Neighbor change over time, but their review scores do not

Every outlet has their own scoring criteria

It should almost go without saying that every site and magazine is different, but for the purposes of rolling together scores on aggregators like Metacritic, there's no allowance made for differing scales.

The venerable Edge Magazine is famous for using the entire scale. A 5/10 game is average. A 6 is good. 7 is very good. 9s and 10s are very seldom awarded. Many other sites barely use scores below 70% unless a game is irredeemably broken.

Without any attempt to reconcile these differences, it's just one more reason that review scores on aggregator sites aren't really useful to anyone, and for readers wanting to parse the numbers for themselves, they need to understand what makes for a high score across multiple publications.

It's almost easier just to read the full review.

Reviews are often written under unrealistic conditions

You're only to look at this spring's mega-hit, Elden Ring for proof of this. Most would agree that it's enormous, at least a hundred hours long. How many people are going to play through a game that huge in a week?

That original review score, no matter how misrepresentative it is of the game that exists now, is what people see when they turn to Metacritic

It speaks to the quality of Elden Ring that reviewers came away with such a high opinion of it while rushing to complete it before deadlines, but a lot of games are better digested slowly. Many are best enjoyed in sessions of an hour or two at a time, over the course of months.

Multiplayer games especially fall foul of this. It's almost impossible for traditional reviews to get a feel for anything online pre-release. This is more of an issue with reviews in general than specifically scores, but those scores are still what sticks the strongest.

What can be done?

It would be a little rude to lay out all this criticism without at least something constructive. There's a lot that could be done to improve the current situation.

First is axing review scores altogether. Several major outlets have already done this, such as Eurogamer and their broad 'Recommended' and 'Essential' tags, and it seems to encourage people to read deeper.

Second is moving away from sites like Metacritic that try to boil down complex opinions into a single number. A number that has even been used to calculate bonuses for some studios -- see the case of Fallout: New Vegas.

But above all, reviews may have to adapt to keep up with ever-changing games. While completely rewriting articles is probably unreasonable to ask, a little addendum near the top, mentioning if a review is strongly out of date, or pointing out key things that have changed would make them far more relevant in the age of streaming and gameplay videos.

Reviews and other long-form critiques are still useful, and probably always will be. But day-one scores and shaky first impressions just don't cut it when games move as fast as they do now.

Alex Nichiporchik is CEO and founder of indie publisher TinyBuild

More stories

Reggie Fils-Aimé on diversity, technology and memeability | Podcast

Former Nintendo of America joins us for our latest episode, available to download now

By GamesIndustry Staff

DAGERSystem acquires Can I Play That

Accessibility non-profit purchases website "to become the largest resource for game-accessibility journalism in the world"

By Brendan Sinclair

Latest comments (2)

Paul Ledbury News Editor / Writer, TheSixthAxisA month ago
I think all of what you said is fine, but it's very narrow view specific to PC games in early access. People forking out £80 for a PlayStation 5 game are not prepared to wait months after they bought it for the game to be finished, the game you get off the shelf is (bar a day one patch) the finished version and therefore reviews can be based on that.

For ongoing games, most good sites do a "Review In Progress" and then go back and shove a score on weeks, if not months later. I did that recently with Destiny 2: Witch Queen, adding a score two months after my initial thoughts.

Anyone not using 5 as average is clearly stupid. There's no getting round that.

As for going back to things, it's simply a case of time. Outlets don't have the time to go back and check and old game to see if it's got better as there are another 20 games this week to review. Nor is there any inclination really because yes, Warframe is great now but since then a hundred other ongoing games have been released and no one has time to keep a track on every single one. What does happen is that if a significant expansion is released (eg Witch Queen), that can be reviewed along with the state of the game today.

As for "talking your time" with a review - literally every reviewer would like that. Every. Single. One. But we can't because we get the codes so late. Some places are much better at dishing things out giving you a good few weeks to relax and enjoy the game - I had three weeks for both Death Stranding and Horizon 2 - but some publishers, who shall rename nameless, drop a AAA console game on sites with barely 48hrs before the game launches.

Yes, kids like the YouTubes. However, as the traffic to written video gaming sites shows, there's still, and indeed an increasing demand, for the written word. Especially from the older gamers.

I think you have a very narrow view on this, sorry. Console games really get demos, betas, and there's no "experience and the community that the game creates.". God of War Ragnarok does not have a official Discord for people to comment on the game and shape it.

Scores still work, especially for one and done console games of which there are still thousands of every year.

Also getting rid of them would annoy the hell out of every PR department in the world, what the hell are the going to slap over their adverts? :)

TC

P.S. We never got a review code for Hello Neighbour *shrug*
1Sign inorRegisterto rate and reply
Julian Cram Quality Analyst (Test), Wicked Witch SoftwareA month ago
Console games rarely get demos, betas
Xbox Gold / Ultimate has demo days and betas. Updates come out almost as fast as they do on PC.
1Sign inorRegisterto rate and reply

Sign in to contribute

Need an account? Register now.