Close
Report Comment to a Moderator Our Moderators review all comments for abusive and offensive language, and ensure comments are from Verified Users only.
Please report a comment only if you feel it requires our urgent attention.
I understand, report it. Cancel

Rebellion: "Nobody here ever bothers with Metacritic"

By Matthew Handrahan

Rebellion: "Nobody here ever bothers with Metacritic"

Tue 17 Feb 2015 11:42am GMT / 6:42am EST / 3:42am PST
PublishingDevelopment

CEO Jason Kingsley on how the rise of Twitch and YouTube has allowed Sniper Elite to thrive in the face of mixed reviews

In the weeks before AOL closed the site for good, Joystiq took a bold step against convention and on behalf of its readership: It ditched scored reviews. Last week, our sister site Eurogamer did the same, and it seems increasingly likely that other sites will soon follow.

For anyone earning a living from reviewing games these will seem like radical times, but the undeniable fact is that the press could no longer ignore the larger forces at work in the industry. As 2014 drew to a close, a string of messy, broken and simply unfinished products cast an unflattering light on the compromises professional critics have to make to get their opinions out into the world on the day of release. The public was angry, and rightfully so.

"Getting a bad product out on time can, and will, damage your reputation as severely as getting a good product out very late"

For Jason Kingsley, the owner of Rebellion and one of the most prominent figures in the British industry - he is, after all, an OBE - that run of deficient AAA releases was a concern, if not exactly a surprise. As the CEO of an independent studio, one that has retained that status through profound change and upheaval, he understands the pressure involved with making and releasing big games. Add to that the grand scale of blockbuster development in 2015, the whims of external shareholders, and the volatility of the online infrastructure, and a flashpoint like this was bound to happen eventually.

"There's a pressure to get the product out there," he says. "But the problem is that getting a bad product out on time can, and will, damage your reputation as severely as getting a good product out very late. It's the devil and the deep blue sea.

"It will have a short-term effect. I don't think it'll take very long for people to think, 'Company X released a faulty product and screwed us over.' I'm not talking 'faulty' as in a few bugs and idiosyncrasies, but something that, on the face of it, is horribly broken. I think we can see it when something really shouldn't have been released. It will have a short-term effect, and I think we'll see changes in the way games have to be made as a result.

"It also damages games in general, because it convinces people that they're all buggy and broken, and they paint every game with the same brush. You can lose your consumer that way."

1

From that perspective, this point of crisis represents a challenge for Rebellion. Sniper Elite, without question Rebellion's most important IP, is aimed at a similar audience to the unfinished games that caused such consternation at the end of last year. If Kingsley fears that consumers may develop negative assumptions about games as a whole, that could easily apply to its tentpole franchise. If there's any seed of doubt, Kingsley doesn't let it show. The over-reaching ambition of the major publishers, he argues, actually leaves the door open for Sniper Elite to expand its audience.

"From a business perspective, Sniper Elite is a slam-dunk certainty. And that's interesting"

"Every Sniper Elite game we've released has debuted at number one, and everybody says, 'God, that's unexpected. It's a sleeper hit. But it's not really a 'sleeper hit' because it sold as well as projects that cost ten times the amount to develop and probably had 20 times the marketing budget. From a business perspective, it's a slam-dunk certainty. And that's interesting."

And it's interesting for two reasons, in particular, the most obvious of which is the glaring disparity in resources behind Sniper Elite relative to its blockbuster competitors. Rebellion has somewhere in the region of 200 employees, but the size of the development team on Sniper Elite v2 - or the size of its marketing spend, for that matter - was of an entirely different order to the typical console retail game expected to be a bestseller. And yet when Sniper Elite 3 got comfortable at the summit of the UK charts in 2014, it held the likes of Watch Dogs and FIFA 15 at bay.

This is clearly a point of some pride for Kingsley, and understandably so. Speaking to Eurogamer in the summer of last year, Kingsley confirmed that Sniper Elite v2 had sold 500,000 units on Steam alone. Add that to console then stand it next to the production budget and it becomes piercingly clear just how influential the franchise could be in terms of the studio's future. Rebellion's long-term goal, Kingsley says, is to be as self-sufficient as possible, creating its own IP and publishing its own games for its own customer base. That future must necessarily be built on success, and Sniper Elite, though still co-published by 505 Games, is laying the bedrock.

The second interesting aspect is highlighted by Kingsley, circling back on the subject of the changing role and shifting responsibilities of professional reviewers. Sniper Elite, you see, has never been the subject of fawning praise from the critics, with Metacritic averages for Sniper Elite v2 and its sequel falling around the mid-60s. There was a time, not too long ago, when that kind of score was enough to scupper a job opportunity or even a bonus, in the belief that big scores are a fundamental aspect of strong sales.

Sniper Elite disproves that notion, its "solid, but a bit disappointing" scores doing nothing to harm its fortunes in the marketplace. Gamers, meanwhile, were almost universally more positive.

"Nobody here ever bothers about Metacritic," Kingsley says. "We think of it as irrelevant, quite frankly. We only concentrate on what the users think, and every aggregate user score has been significantly higher than the aggregate professional score. We care about the people who are spending their money, and whether we're happy that we've made a good game. The acid test isn't somebody's abstracted number.

"Every aggregate user score has been significantly higher than the aggregate professional score"

"Professional reviewers have a very difficult job, because they cannot see a game from the perspective of somebody who's paid money for it. Because that's their job. Your average player who buys the game is almost obligated to try and enjoy it. They're hoping this thing they've paid for is good, and if it's crap they're very, very disappointed indeed and they probably won't buy another game from you. I'm not saying professional reviewers try not to enjoy games, but that's what they do during the week as a profession, with all the pressures and deadlines that come with it."

With all of this in mind, the apparent bravery of the press in discarding review scores takes on a different complexion. The time when that practice adequately served either the product or the audience has long since passed, and it shouldn't have required such a dismal period of broken releases to finally tip the balance. Indeed, that insistence on abstract scoring systems has anchored the way the press examines and evaluates games firmly in the past, even as the internet demonstrated, time and again, just how much more creative, detailed and, crucially, useful the form could be.

2

On a personal level, I purchased Far Cry 2 not because of its review scores or even brand recognition, but because of the blogger Ben Abraham's accounts of playing the game while deliberately enforcing permanent death. The same is true of The Sims 3, and Robin Burkinshaw's wonderful role-playing experiment Alice & Kev. That was 2009, and there was nothing even remotely similar to that being generated by the press. Despite the new freedoms afforded by the fluidity and immediacy of the internet, commercial games coverage has remained grounded in a carousel of previews and reviews - a structure created to serve magazines.

"Professional reviewers have a difficult job, because they can't see a game from the perspective of somebody who's paid money for it"

As someone who rode that same carousel for five years, I can say with confidence that the indolence shown by the press in this area is only partly due to the sluggish nature of large organisations. The simple fact is that reviews - in their classic form - are the low-hanging fruit of games coverage; a rigid, undemanding format enlivened by the addictive tang of emphatically stated authority. The alternatives, though well within reach, were simply more difficult to contemplate and execute. That's how self-starting individuals like PewDiePie and TotalBiscuit managed to steal a march on those with the equipment, resources and reach to get there first. Commercial games coverage will change irrevocably over the coming years, and that will be down to YouTubers, Twitch and the long overdue recognition that review scores are little more than dead weight.

For Kingsley, this is the landscape in which companies like Rebellion have always wanted to operate. One where the lines of communication aren't mediated by a handful of paid professionals, the common view of any given game orchestrated by their personal tastes, unencumbered by the need to actually pay for the product. For Sniper Elite, with its laser focus on polishing a handful of specific, sticky and shareable game mechanics, that made all the difference.

"The greatest value for us as digital publishers, if you like, is in embracing YouTube and Twitch and the normal people being seen playing our games," Kingsley says. "It lets you see what the gameplay is like, and make a decision on whether you like that and want to play it. You might not actually care whether it didn't seem totally original to one person, or that the story was a bit crap.

"That new approach has taken over."

From GamesIndustry.biz Recommendations by Taboola

14 Comments

Tom Pickard Founder and Creative Director, Knifey Spoonie Games

20 25 1.3
I think it's a fantastic approach. Just like Eurogamer dropping review scores.. A "number" to determine a products worth is almost pointless. There are too many variables. Also like any entertainment product you might enjoy something whilst the reviewer might not. A good reviewer however will coax out what is good about the game and what others might enjoy or find frustrating but might still give a game a 6 because it's not to his preference.

focusing on making your game fun or unique is the best way to get the most out of your product and If you do it well you'll find an audience despite the Metacritic. Focus on the features that stand out to your audience. Focus on entertainment and quality. If you also get a great metacritic as well.. then awesome

Posted:A year ago

#1
Score! (for no scores that is) :)

Posted:A year ago

#2

Pete Thompson Editor

236 164 0.7
Good article, and on good Jason Kingsley for saying that they don't bother with Metacritic..! I've not looked at Metacritic scores for years, mainly because they only take a game into account if and when It's released in the U.S.A and don't consider games releasing in other regions..

As for review scores, we dropped ours early last year..

Edited 1 times. Last edit by Pete Thompson on 17th February 2015 6:27pm

Posted:A year ago

#3
Scores are good measurement for readers to quickly get summary of the game. That has not changed.

Posted:A year ago

#4

Craig Page El Presidente, Awesome Enterprises

508 346 0.7
They should care about Metacritic, and even their score on Steam's user reviews. I trust the opinions of actual gamers a lot more than the opinions of people who just write about games for a living.

Posted:A year ago

#5

Phil Elliott Project Lead, Collective; Head of Community (London), Square Enix

185 58 0.3
I've always thought that one of the problems with Metacritic was that it's lumping in apples with pears - and while there's weighting, that adds another layer of subjectivity. Can you really call a Eurogamer 8/10 the same as an IGN 8/10? And if not, how do you compensate for that effectively?

I agree completely that having a metascore is useful, but as soon as it became a barometer for actually important things, such as developer bonuses, etc, that made me worry a lot. But then, what do you use? Sales used to be the barometer, but what about that critical success that just didn't sell?

And user reviews are an important part of judging quality - except when there's a controversy and a game or developer becomes a target for something that's not to do with the game's actual quality. Or the system is even open to scamming by a developer or publisher to buy fake reviews.

So nothing's perfect in every situation; personally I like to look at a range of critic scores, user reviews and sales numbers... but I'd hate to be the person to work out how to benchmark dev bonuses fairly...

Posted:A year ago

#6

Jessica Hyland Artist, Turbulenz Limited

464 2,091 4.5
Individual developer bonuses should be based on individual developer performance(and however you measure that internally is then a question of good management rather than leaving it up to a third party website), not how their game performed in the market. Most people working in a studio environment with tens or hundreds of colleagues have little chance to directly impact on the sales of a game unless they're very senior figures. It's not really very fair for the bonus of say, an excellent programmer who happens to be working on a game with poor art direction to be impacted negatively by something the programmer has absolutely nothing to do with. Vice versa, artists working on a game that's visually stunning but plagued by gameplay bugs end up being punished for problems outside of their hands.

If you must have whole-team bonuses based on the game's performance(on Metacritic or otherwise...), make those a small addition to individual performance bonuses. The alternative can be very demotivating, in my experience.

Personally, I think far too much importance - by consumers and publishers both - is placed on Metacritic and numerical scores.

Posted:A year ago

#7

Anthony Gowland Consulting F2P Game Designer, Ant Workshop

320 1,400 4.4
They should care about ... their score on Steam's user reviews.
The quote explicitly states that they do care what their user ratings are.
We only concentrate on what the users think, and every aggregate user score has been significantly higher than the aggregate professional score. We care about the people who are spending their money, and whether we're happy that we've made a good game.

Posted:A year ago

#8

Matthew Handrahan European Deputy Editor, GamesIndustry.biz

143 159 1.1
Thanks for pointing that out Anthony. There are times when it feels like the article is just propping up the headline. Admittedly, though, there are also times when that's kinda true. :)

Posted:A year ago

#9

Carlos Brandão Director, Game Rental

16 7 0.4
I think the assessment by Metacritic.com are very hard, very tough! The game or the movie needs to be perfect to get a scoring high. Anyway, I think it influences indeed rather the end consumer, who can consider that notes a good measure of quality of games or movie. When u have multiple versions of the same game, the comparison of notes weighs enough.

Edited 1 times. Last edit by Carlos Brandão on 20th February 2015 1:14am

Posted:A year ago

#10

Greg Wilcox Creator, Destroy All Fanboys!

2,603 1,676 0.6
"Professional reviewers have a very difficult job, because they cannot see a game from the perspective of somebody who's paid money for it."

Not always true. In the past, I've gone and paid for games some PR folks wouldn't send me for whatever reasons and reviewed them fairly (for better or worse). Even if I do get a code or physical product, I tend to write reviews not for myself, but for people who may be interested in buying the game. That's how it should be done, but it seems that a lot of ink goes to ego central these days (particularly on paid channels where "personality" rules over constructive criticism - yuk.)

Posted:A year ago

#11

Morville O'Driscoll Blogger & Critic

2,017 2,357 1.2
And then there's the other view, that someone who's paid money for something will try and find reasons to like it, as a means of justifying the money spent. Whether that be a consumer buying something blind on Steam, or a reviewer who's fronting their own money so they can post their review on their website.

Posted:A year ago

#12

Adam Jordan Community Management/Moderation

148 144 1.0
There's two sides to every story, a good and a bad.

Steam reviews for example have a good side that you are getting a direct source from the users themselves but that in itself is a double edged sword. Reason being is that people will use sarcasm when rating a game, such as "10/10 I recommend this game if you LOVE a ton of bugs"

Metacritic on the other hand has the issue that you have not got a direct source to the users (Sure you have user reviews as well but they come under the same scrutiny as above) however you do gain a more professional stance upon the matter.

As a consumer and gamer, I never look at Metacritic, it may take the average of all pro reviews but even back in the day of Gaming Magazines, well before Metacritic, I would scoff at some scores given to games because personally I either didn't like them or felt a game was underrated.

For me, I listen to friends or people whom I believe have a good sense of things that would be fun. SO what's the problem with Metacritic or even user reviews?

Well, User reviews are prime to become lynch mobs or "follow the leader"...example being CoD vs BF, I believe when Black Ops was released alongside Battlefield Bad Company 2, Black Ops was subjected to a lot of down votes not just because the franchise was going stale or because it was on an annual dev cycle but also because of their rivalry. It fired up the fanboys of both franchises to go head to head.

Metacritic pro reviews however are under scrutiny (At least back in the day) because it couldn't account for those that were allegedly "paid off" to give games nothing below 9.0. In other words with most consumers, it lost their trust, even though it wasn't directly Metacritic's fault.

Also to note, what Morville mentions rings very true. People will go to any means to justify the money they have spent on something.

My point is, neither side (pro or user) gives a clear indication of what a game deserves, plus over the years some pro reviews give the sense that "This is the only opinion you need, listen to no one else". Likes and dislikes are very personal, "One man's trash is another man's treasure" and that you can never put a score upon.

Posted:A year ago

#13

Paul Johnson Managing Director / Lead code monkey, Rubicon Development

1,219 2,667 2.2
Steady how you go with buying into the PR line here guys. "Company with poor metacritic says ignore metacritic" is all I'm getting, and I liked the (first) game. I'm sure if they had a score of 98 you'd be hearing about it much more.

Posted:A year ago

#14

Login or register to post

Take part in the GamesIndustry community

Register now