Close
Report Comment to a Moderator Our Moderators review all comments for abusive and offensive language, and ensure comments are from Verified Users only.
Please report a comment only if you feel it requires our urgent attention.
I understand, report it. Cancel

Warren's Mailbag: Publishers using Metacritic as a "weapon"

Warren's Mailbag: Publishers using Metacritic as a "weapon"

Wed 19 Jun 2013 2:16pm GMT / 10:16am EDT / 7:16am PDT
MediaPublishingDevelopment

Warren Spector answers your questions about the problems with the reviews aggregation site and the industry's reliance on a flawed system

Thanks to all of you who responded to my second column for responding so interestingly and for providing me with so much to think about. I got more than enough comments, on the GamesIndustry International site and Facebook and elsewhere that another “Warren's Mailbag” seems justified.

As in the last mailbag, I've aggregated similar comments and tried not to quote anyone specifically. If you notice your exact words, well, congratulations, I guess?

Anyway, here goes...

People who develop and publish games really want some sort of assessment of their work, maybe even validation from some presumably “objective” party - whether that's critics or players or both.

Hm. I guess some developers want or need validation of their work from some outside party. Maybe as a guy who doesn't read reviews I'm an anomaly. (However, to be fair, plenty of people tell me what reviewers and fans are saying!)

The more interesting part of this sort of comment is the use of the word “objective.” Several commenters used the word and I found it troubling. Fact is, there's no such thing as objective criticism or evaluation. Pretending there is may be part of the problem with the current state of the art in games criticism. Reviewers HAVE to acknowledge their own biases and make sure readers, listeners or viewers understand them.

But where the word “objective” really worries me is in regard to Metacritic and other attempts to quantify a game's quality. What aggregation sites do is take subjective data (reviewers' opinions), weed some out by running the reviews through an unknown inclusion algorithm, translating each remaining review's score - “out of 10,” “out of 100,” “out of 5 stars,” “A-F” or “we don't give scores” - into a single scale somehow, weighting the now invalid scores according to some unknown algorithm and then averaging them out... I mean, when you look at it that way, isn't it ridiculous?

All developers control is the quality of their games and, therefore, need a definition and a measurement of quality.

Couple of comments here. I've already spoken to the idea that developers need a definition and measurement of quality. Some of us don't particularly want it, let alone need it. Or, more to the point, there are ways of getting such measurements that are useful and some that are not.

"The fact is it's a tool - a weapon - wielded with a heavy hand by publishers"

The second point that often went along with this sort of comment is that developers control the quality of their games. That's certainly true, to an extent. But it's important to remember that making a game is a team effort and, often, there are many people working on and making decisions about a game who are outside the developer's purview - think marketing people, PR people, publisher-side producers, executives... I could go on and on. I've always said that the determinant of a game's success (in terms of achieving the developer's goals!) is how much time for tuning and debugging the publisher gives you from the time you hit Alpha (i.e., fully playable, not-fun game). Control of quality falls fully on the developer's shoulders only in the indie world (and I bet a bunch of indie developers would argue I'm even wrong about that!).

To ignore reviewers and sites that aggregate review scores is to risk falling prey to uncontrolled ego and arrogance. It's important to believe in a vision, but not to the extent that feedback reveals flaws in that vision. You need balance.

Yeah, well, okay, maybe... I certainly know a lot of egomaniacs (myself probably included) but I haven't been accused of arrogance (today...). Seriously, there's no doubt that you have to find a balance in staying true to your vision and allowing feedback to reveal flaws that lead to reevaluation. But there are ways to get that feedback that are useful and ways that are not.

Any developer who doesn't get his or her game in front of actual players - early, often and continuously - is on a very dangerous path. Watching people play your game is critical. (Heck even the kind of focus tests publishers love to pay for can have value.)

Any developer who doesn't get trusted developer friends playing his or her game is equally foolish. There's huge value in the thoughts of people who truly understand the development process and can evaluate (and eviscerate) your game with sensitivity to where you are in that process.

What's dangerous, if not outright useless (and dangerous), is allowing backward-looking data to divert you from your course. Anyone who charges thousands of dollars to predict review scores of work in development is offering nothing of value. Anyone who drops in for a “deep dive” of a day or two can't have the context to provide useful input. Anyone who thinks you can look at the schedule of past releases, review scores of comparable games or the aggregation thereof is pulling your leg. Laugh and move on.

Reviewers should explain how they came to the conclusions about a game that they did and recognize that his or her opinion isn't absolute and may not match that of readers, listeners or viewers.

Several readers commented on the need for reviewers to explain not just what they think about a game but why they think it. That's certainly true. But the related idea that opinions aren't absolute and may not reflect that of other people? There's gold there. That's one of the key points I was trying to make (in way more words!). As a reviewer, your job is to help a potential buyer determine whether he or she will like a game - not whether you did. Put those two ideas together and you get something useful: Explain yourself so others can decide for themselves.

In the age of Big Data (or even small data) there's a tendency to believe that only that which can be quantified has any value. Game reviews, by their nature, are opinions and therefore, not quantifiable. Several people commented on the 21st century need to quantify everything - and to discount anything that can't be quantified. I have a funny feeling people are going to look back on this idea, fifty years from now, and laugh. A lot. But even looking at the situation today, look at the results of all the testing and aggregating and data-munging: Most games fail, commercially, at a rate that should embarrass anyone in the game-testing business. Many that succeed do so in the face of significant obstacles. Speaking personally, the worst reviewed games I've worked on are the best-selling. Several of the most data-driven-design proponents are in serious trouble. You can keep your data, thank you very much.

Clinging to genre conventions - and reviewing conventions institutionalized by review aggregation sites - limits developers to what came before and stunts our growth as a medium. Or... there's no limit on game creativity, a fact reflected in the scores of indie games and commercial games that fly in the face of convention.

Reading some of the comments was pretty funny. A lot of folks said that review aggregation held us back, creatively, by institutionalizing the known conventions that lead to high scores, at the expense of risky ideas that might lead to lower ones. I kind of took that position in my column. But several people pointed to the high scores of several hugely risky and innovative games as proof that I was wrong.

Mea maxima culpa. I think I was wrong. Anecdotally, there really doesn't seem to be much of a connection between review aggregation and the dominance of innovative games. (Which isn't to say we couldn't use more innovative games!) Anyway, remove one sin from Metacritic's list. There are still plenty left...

The relationship between publishers and reviewers represents a conflict of interest. Things like junkets and ads and schwag essentially “buy” review scores.

Whew. This is a tough one. I've always wondered whether junkets and ad buys and flashy E3 booths and so on actually influence game reviewers. Honestly, I've never seen any evidence of that. Doesn't mean it isn't happening, but until there's some evidence (anyone got any?) I'm going to think the best of people - even game reviewers...

Contracts use Metacritic thresholds to determine developer bonuses? If your threshold is 85 and you get an 84 - no bonus?!

I was surprised how many people didn't know this or didn't want to believe it. It's all too true and all too common. You can say until you're blue in the face that Metacritic is designed to assist players in finding games they want to play. It may have started out that way. The people who run it may think it's still that way. The fact is it's a tool - a weapon - wielded with a heavy hand by publishers. Developers sign away bonuses on Metacritic scores for games that don't even exist as a single page concept doc. If you believe in the validity of review aggregation that may not sound like such a bad deal - the people who pay for a game should base bonuses on some measurable standard. But if you (a) don't believe in Metacritic's validity or (b) know how freakin' hard a team has to work to make even a “bad” game, that one-point Metacritic score difference means a lot.

The variety of games being rated on Metacritic is too broad to result in valid, useful data.

Frankly, I'm a little ashamed that I didn't think of this myself. We would never compare Woody Allen's movies with Michael Bay's. Why does it make sense to compare Journey, The Walking Dead, Call of Duty 72 and the latest Madden? Does anyone really care that one got a 7.6 while another got a 9.2 or whatever? The silliness is self-evident. Given that players and publishers take aggregation seriously, it's also damaging.

"Simply liking games and being able to construct a sentence doesn't prepare you for a job as a reviewer and certainly not for a deeper critical role"

How are reviews selected for inclusion and how are the numbers used?

Great question. We should all insist on knowing how a review gets included in or excluded from Metacritic's rankings and how much each review counts for. It's ridiculous that this information isn't available. It affects people's pay, players' ability to determine the validity of the data they're getting, and the future of game franchises. If we're not going to be told how the numbers are generated, why should we pay any attention?

I know of games where bad scores were included instantly - even if they came from a random-guy-with-a-website - while good scores from major media outlets were nowhere to be found. One commenter opined that outlier reviews would be balanced - or revealed to be irrelevant - by the averaging with higher scores. This is only true when the inclusion decisions are reasonable. With Metacritic we just don't know.

Film criticism is relatively simple since a movie runs just a couple of hours making even multiple viewings possible before a review gets written and certainly before a deeper critical piece comes out. Games take anywhere from five to 100 hours, with most in the 15-20 hour range, making even one playthrough difficult before a review appears.

There's certainly some truth to this - films ARE easier to analyze than games. Having said that, most of the game reviewers I know DO play games to completion before writing about them. I don't think the problem is completion of games so much as the lack of training most reviewers have and a lack of anything like a coherent critical vocabulary for discussing games.

As I said in the column, the job of a reviewer isn't to say, “This is good” or “This is bad.” It's to convey to a reader/listener/viewer whether he or she will like a game. It's to express a consistent critical (dare I say it?) philosophy so readers can decide for themselves how they feel about the game elements singled out by a given writer. Simply liking games and being able to construct a sentence doesn't prepare you for a job as a reviewer and certainly not for a deeper critical role. We'll get there, eventually. I just don't think we're there yet.

Game designers spend too much time pontificating on stuff. The people with power, on the other hand, pat them on the heads and tell them to get along. Game designers should form a guild and make demands if they really want to change things.

I had to run this comment verbatim!

The fact that game designers spend too much time pontificating is undeniable. We like communicating. A lot. No excuses. If it bothers anyone, just ignore us. We're used to it.

The comment about people with power was great. The only problem with it is the idea that we get patted on the head - swatted on the butt is more like it.

"If Metacritic is the best we have, we're in a world of trouble. Again, though, it's not so much that 'Metacritic sucks'... We just have to acknowledge what it is, push it to be better and then use it appropriately"

Starting a guild? I'd love that and there have been talks about it over the years. Maybe I'll do a column about that idea some time. Yeah. That'd be interesting...

Since it combines reviewer and gamer scores as well as links to individual reviews, Metacritic allows people to make up their minds. It's the best we have.

Here's the deal. I get that Metacritic can be used by people to decide what games to buy or not buy. And in this fast-paced, helter skelter world we live in, shortcuts that allow us to make decisions without thinking too hard are appealing and seductively attractive. If you don't want to think, by all means check the scores on Metacritic and get on with your life. Just don't be surprised when the games you buy don't end up working for you.

As far as player reviews go, it's certainly good to have them - good in the abstract, at least. However, some commenters, you'll recall, complained that reviewers didn't complete games before reviewing them (which I choose not to believe) - what do you think players are doing? I bet most of them haven't even tried the games they're scoring! At risk of letting loose the hounds of hell, a lot of people on the Internet are just interested in attacking stuff, usually with copious cursing. (Oh, you haven't noticed that? Well, it's true.) I'm not sure player ratings are any more accurate than professional review scores.

And if Metacritic is the best we have, we're in a world of trouble. Again, though, it's not so much that “Metacritic sucks,” though it may sound like that's what I'm saying. We just have to acknowledge what it is, push it to be better and then use it appropriately. Right now, I don't think we're doing any of those things.

The level of “games literacy” is low, something that can only be addressed through criticism rather than consumption. If gamers demanded more nuanced evaluations of games, reviews would get better and Metacritic would be less important.

Oh my do I agree with this. I agree, to an extent, with those of you who said criticism can and should play a part in addressing this. However, while reviews-as-criticism are important, I think the more academic sort of criticism is as important, maybe more so.

Criticism doesn't have to be only a consumer service. It can explain how games work. It can change the way we think about games. It can, ultimately, change the kind of games that get made. There are some universities that get this and are making sincere efforts to address the problem. And I can tell you that, as we develop the curriculum for the Denius-Sams Game Design Academy at the University of Texas, I'm going to be working hard to ensure that we include serious critical analysis along with hands-on game development.

Rather than think of all games as “commercial art” acknowledge that there are some that are purely commercial (designed to generate a profit) and others that are purely art (designed to express a personal vision). I'd never deny that games are made by all sorts of people and institutions for all sorts of reasons. I sometimes wish I lived in a world of pure indie creativity and I bet a lot of my peers do, too. I was talking only about games that are commercial in some way. I applaud the artists and wish them well. 'Nuff said.

I don't think there is a way to predict or guarantee success.

Can I get an amen, brothers and sisters?!

17 Comments

Dan Howdle Head of Content, Existent

281 814 2.9
For what it's worth, Warren, I hate Metacritic too. Criticism is an art more than it is a science – since it sits well within the inner bullseye of subjectivity – and art cannot and should not be homogenised. It devalues what we do and, believe it or not, some of us are responsible and professional enough to be bothered by that.

Edited 2 times. Last edit by Dan Howdle on 19th June 2013 5:44pm

Posted:A year ago

#1

Todd Weidner Founder, Big Daddy Game Studio

420 1,000 2.4
Over 7 billion people on earth, and every single one of us is capable of forming an opinion. It cracks me up that some website/magazine hire some person and then declare and act as this act of hiring ads some magical credibility. As if their opinion magically speaks for the masses somehow. Baloney,

Listen, I have no problem with someone taking the time to thoughtfully express their thoughts and opinions on a game/movie etc but all to often these same people then arbitrary narrow down that opinion to some gradeschoolish grading system which totally destroys any thoughtfulness and nuisance that opinion may have had. Furthermore for any corp or industry to then go and act as if this opinion carries some kind of populous weight to it? its lazy and ridiculous.

Arent we adults here? arent we able to discuss our opinions without having to resort to childish grading?

Edited 3 times. Last edit by Todd Weidner on 19th June 2013 5:48pm

Posted:A year ago

#2

Dan Howdle Head of Content, Existent

281 814 2.9
There is no such thing as magical credibility, but there is such a thing as credible qualification. Professional is what professional is. Speaking as a long-time editor I can tell you that finding somebody to work as a credible staff writer is crushingly hard. The skillset requires exceptional writing ability, the ability to generate novel and interesting ideas, a fine aesthetic eye, a deep inside knowledge of the games industry, and a personal history of having played all the big games on all the big formats. It gets very specific and whether it fits your particular notion or not, these people are rare.

Can anyone at all can perform surgery? Because we all have hands.

Edited 1 times. Last edit by Dan Howdle on 19th June 2013 6:00pm

Posted:A year ago

#3

Todd Weidner Founder, Big Daddy Game Studio

420 1,000 2.4
Really equating having the ability to do surgery with having the ability to play a game and form an opinion as similar? really?

Sure having some writing skills may be a professional quality, but being able to write well has nothing to do with a validity of an opinion.

PS. To be clear, I do have a great deal of admiration for journalism, and I do agree wholeheartedly that in order to do journalism the writer must be knowledgeable in the subject matter, with that said, I just tend not to lump in a " review" with journalism.

Edited 4 times. Last edit by Todd Weidner on 19th June 2013 8:43pm

Posted:A year ago

#4

Tim Carter Designer - Writer - Producer

582 322 0.6
Mr Spector should read "Anatomy of Criticism" by Northrop Frye before he declares there's no such thing as objective criticism.

Posted:A year ago

#5

Dan Howdle Head of Content, Existent

281 814 2.9
Really equating having the ability to do surgery with having the ability to play a game and form an opinion as similar? really?
Learned skills and experience are the same no matter what the vocation (or how highly that vocation is regarded); time, effort, will, skill and meticulous attention to detail are the operating factors. I pluck 'surgeon' from the ether because it's an extreme example of the principal, not because it's comparable to critical writing and thinking. I do contest that good writers are a great deal rarer than good doctors, but that's another conversation.
Sure having some writing skills may be a professional quality, but being able to write well has nothing to do with a validity of an opinion.
But writing ability combined with professional expertise does. Somebody – let's call him Bob – who picks up the latest Call Of Duty as their first ever videogame and declares it the greatest game of all time has an opinion, but it's an opinion that's useless to anybody but himself. He can share that opinion but nobody cares, nor should they. And it's a sliding scale.

What do you do when you want to find out how good the latest World Of Warcraft expansion is? Do you want to hear that from someone who a) Can write (communicate their opinion effectively and entertainingly [because you don't want to be bored reading it either]) and b) Has spent the last seven years of their life playing WoW, has interviewed the developers, is familiar with the innermost complexities of its mechanics, understands his or her readership and provides insight you cannot garner even from playing the thing yourself? Or do you want to hear it from Bob the COD guy?

If your argument is that all opinions are equal then I respectfully disagree. Both research on the subject (as Tim points out with but one example, above), and common sense all say no, as does that doctor/importance of expertise analogy you don't like. Bob thinks it's not stomach cancer, so you'll probably be fine.

If you're saying all opinions are subjectively valuable, then I agree; it's just that those opinions lack the expertise, the insight, and the skill to hold any value beyond the confines of that individual's skull.

Edited 7 times. Last edit by Dan Howdle on 19th June 2013 9:30pm

Posted:A year ago

#6

Todd Weidner Founder, Big Daddy Game Studio

420 1,000 2.4
it's just that those opinions lack the expertise, the insight, and the skill to hold any value beyond the confines of that individual's skull.
and again I disagree, when just speaking of reviewing an experience ones has/had playing a game.

example, you say
Somebody – let's call him Bob – who picks up the latest Call Of Duty as their first ever videogame and declares it the greatest game of all time has an opinion, but it's an opinion that's useless to anybody but himself. He can share that opinion but nobody cares, nor should they. .
I say your are missing the point. Bob gives a very valid opinion and perspective. Coming from the vast majority of the world who have yet to play a video game, Bob states that in his opinion as a first game tried Call of Duty is a blast. This review may be very useful to those new to the hobby.

example 2- If I m a 12 year old girl gamer, you really think a review from some 38 year old male tech geek is more valid to me than say a review from another 12 year old girl?

Reviews are opinions, that's all, and everyone has one. There is nothing special about one over any other for the most part. Reviews are not some sort of expose' on the industry, reviews are nothing more than relating to another person your opinion of an experience you just had with a product.

and to be honest if opinion writing was so difficult blogging wouldnt have been able to take off the way it has, but hey since writing is harder than surgery in your opinion, I guess we should all get ready for do it your self surgery kits soon, since blogging is done by millions, home surgery should be done by even more /jk :P

Edited 8 times. Last edit by Todd Weidner on 19th June 2013 10:46pm

Posted:A year ago

#7

Axel Cushing Writer / Blogger

107 132 1.2
I'm a reviewer, and I hate Metacritic.

If there is any virtue to Metacritic, it's that of an aggregator. If there's a site that I particularly like, but I'm having a hard time finding their review, I can go to Metacritic and it's right there. The downside is that what I consider a minor but nice utility is seen by publishers as some kind of infallible crowdsourced metric which cannot be challenged, discussed, considered, or qualified. It is data. Data is not wrong. Ergo, Metacritic is not wrong.

The funny thing is that the publishers look at the big score in the box, the official reviewer score, and I'm morally certain they pay not one iota of attention to the other score on the page. The one in the circle. The one from the fans. Far as publishers are concerned, it seems to be "Sod the proles! The ignorant sheep don't know what they're bleating about!" And that, I think, is not a failing of Metacritic, but those who misuse Metacritic. The data is there. There may be some false data, some trolls trying to shank the score, some fanbois who try to pump the score, but it all averages out in the end. I think that's probably the better score to be paying attention to, within reason.

I was at E3 last week, went to Activision-Blizzard's booth, sat down with the new game director for Diablo III. I don't think he had any idea that he was talking with the one person who not only panned the first iteration of the game he'd inherited, but was probably going to be the hardest sell of the show. I actually did like what he had to offer. But the series still feels tainted in my eyes because of the shenanigans from the PC release. Since they've proven the game doesn't actually need the auction house, or the always on connection in order to function as a game, it's probably going to taint it even further. Point is, nobody connected the scathing review I gave the original game a year ago to my presence there this time. There's a tremendous degree of cherry picking in the data. It's all about the little average box score.

As long as all that the executives and marketing wonks are interested in is that average box score, I will hate Metacritic. Not for what it is, but for what it is used for.

Posted:A year ago

#8

Dan Howdle Head of Content, Existent

281 814 2.9
example 2- If I m a 12 year old girl gamer, you really think a review from some 38 year old male tech geek is more valid to me than say a review from another 12 year old girl?
Proving exactly the point that I'm making: that all opinions are not equal. Which is why, in a court of law, expert testimony is commonly called upon to bolster or weaken the case for a particular point or dispute. That 12 year-old should not be called upon for her blood-spatter analysis.

If you believe that 'game reviewers' or any other type of critic for that matter possesses no greater insight or technical ability to express said insight than a 12 year-old girl, you need to have a rethink. People want to know what Mark Kermode thinks because he is an expert on film. People want to know what Jay Rayner thinks because he is an expert on food. That expertise does not derive itself solely from watching films or eating food. It comes from deep knowledge of the subject matter that is both rare and unusual.

If your point is that anyone can blog, then sure, that's true, but the currency of RockPaperShotgun over Bob's games blog highlights the flaw. It's a cacophony of crap out there; a formless soup of misinformation and ignorant ranting, and that's why people coalesce around the sites and print publications where they know they will receive well-written, expert opinion.

After all, why are you here on GI.biz?

Edited 4 times. Last edit by Dan Howdle on 20th June 2013 7:54am

Posted:A year ago

#9

Greg Wilcox Creator, Destroy All Fanboys!

2,195 1,169 0.5
@Axel: Opinion here. I see one very small problem with your post - very small, but interesting:

Maybe your future review of any console version of Diiablo III should focus on what works FIRST, not what's not in there that didn't work so well on the PC. Scathing is good when done right (constructively), but only if it's fitting the product in question and not an earlier version on a different platform.

At least that how I'd tackle the review.

As for people who want to buy the game when it hits? Guess what? If most of them wanted to do so on PC... they'd have done it, I'd bet. Blizzard is wisely expanding its user base with DIII by allowing NEW users to play it, not giving the "exact" same experience to console owners and hoping for the best.

It's almost like reviewing a Wii U "port" that has more (and improved) features and wasting column space talking about another version that a Wii U-ONLY owner won't care about much because they'll want to know if the game in its newer iteration is fun on the console they own.

Just a thought...

Posted:A year ago

#10

Todd Weidner Founder, Big Daddy Game Studio

420 1,000 2.4
If you believe that 'game reviewers' or any other type of critic for that matter possesses no greater insight or technical ability to express said insight than a 12 year-old girl, you need to have a rethink
again, it is you that are missing the point, again I ask
If I m a 12 year old girl gamer, you really think a review from some 38 year old male tech geek is more valid to me than say a review from another 12 year old girl? think about that for awhile. We actually agree that not all opinion is equal, since the validity and importance relies on the readers needs, perspectives, and wants. In my example the 12 year old is much better served by a review of another 12 year old.
After all, why are you here on GI.biz?
hint.. I am not here for the game reviews

Edited 3 times. Last edit by Todd Weidner on 20th June 2013 2:26pm

Posted:A year ago

#11

David Serrano Freelancer

300 272 0.9
"People who develop and publish games really want some sort of assessment of their work, maybe even validation from some presumably “objective” party - whether that's critics or players or both."

What designers and developers need is real time feedback from people who will view their work from a different perspective. Because when people work on projects for an extended period of time, they frequently lose perspective on their work. What I learned from personal experience was when you start to lose perspective on your work, you also lose the ability to see obvious problems staring you in the face. And based on my experience, the feedback I received from non-experts who had no personal or financial involvement in a project was always more valuable than feedback from co-workers. Because non-experts viewed the work from the perspective of the consumer. While co-workers tend to rationalize and reinforce why problems or mistakes are acceptable based on conventional wisdom.

Posted:A year ago

#12

Paul Johnson Managing Director / Lead code monkey, Rubicon Development

925 1,569 1.7
Metacritic isn't about developers or publishers, it's about the punters. And whilst I fully ackowledge that invidivual opinions can vary a lot, an "average" opinion is a powerful thing indeed.

If a game has a rating of 90 you might not like it, but it's been done well and has no flaws and is worthy of looking at
If a game has a rating of 40, then it's shite. Even if you like games in the same genre, don't buy this one.

And that's all the punters need or want as a quick look see. Presumably if they're still not sold by the 90+ they can go root out some of those reviews and get a feel for whether they might give it a 90 themselves and go buy it.

(disclosure, most of our stuff has a very high rating. But if everyone gave our stuff 40% i'd probably hate it...)

Posted:A year ago

#13

Axel Cushing Writer / Blogger

107 132 1.2
@Greg
Chances are I won't be getting that review assignment in any case. Generally, when a title shows up on different platforms, different writers get specific versions. And I'm honest enough to admit that my editor probably knows that I might not be entirely open-minded when it comes time to dole out the review copy of the game. All of that said, the 360 version that I played around with for a bit during the presentation wasn't half bad. Had a very Gauntlet-like feel to it.

The larger point goes back to the selective amnesia/tunnel vision that seems to infect publishers when it comes to Metacritic. As long as the average of official reviews is above their desired level (usually 85), they're perfectly happy. There doesn't appear to be any sort of critical analysis of what else is on there. There doesn't appear to be any sort of admission that the official reviews have a point when the reviewers point out a problem. No acknowledgment that there are fans of the game who are unhappy enough about a title to post on Metacritic in the first place. Or, take the inverse: a game which might miss the magic number publishers are looking for from the official reviews, but the fans might call it a resounding success. The developers might take some comfort that they have fans, but when their bonuses are tied to the official review scores only, it's cold comfort.

Posted:A year ago

#14

Greg Wilcox Creator, Destroy All Fanboys!

2,195 1,169 0.5
@Axel: you're spot on there, as it's a common thing to look at any games message board and see plenty of "metacritic gave it a..." posts which are insane, as Metacritic doesn't "give" anything other than a number created from other numbers rounded up or down that never, ever show a game's true worth to those who play them for more than a compiled set of scores...

Posted:A year ago

#15

Warren Spector VP and Creative Director, Junction Point

1 1 1.0
Hey, Tim Carter, what makes you think I haven't read Northrop Frye? heck, I read Anatomy of Criticism when I was in grad school. That'd be about 35 years ago... For a while, I believed in the whole objective criticism idea, as you do. I respect your opinion - in fact, I defended it in many honorable intellectual battles with friends and fellow students back in the day.

But over the years, I came to believe the concept was flawed, so I rejected it (as you know, since you commented on it). We can certainly agree to disagree, but talk to me in 35 years and we'll see how you feel about Anatomy of Criticism. (You know, there are times when it's good to be an old guy!)

FWIW, your comment convinced me it's time to go back and read ol' Niothrop again, so thanks for that.

Edited 1 times. Last edit by Warren Spector on 22nd June 2013 12:15am

Posted:A year ago

#16

Eyal Teler Programmer

93 99 1.1
If I m a 12 year old girl gamer, you really think a review from some 38 year old male tech geek is more valid to me than say a review from another 12 year old girl?
In many cases, yes. If you're a 12 year old girl who's into adventure games, the review of a 38 year old male who likes adventure games will mean a lot more to you than a review from a 12 year old girl who thinks that nothing is better than The Sims. Similarly, a review from a 38 year old saying that a game has an interesting plot but is bogged down by overdrawn combat may be more relevant than a 12 year old girl saying that a game rocks "because Vaan is so hot". Saying that a 12 year old is a better reviewer for another 12 year old is as silly as implying that reviews done be a women or someone of Chinese origin are inherently less relevant to me than those of a guy of Eastern European descent. We're all individuals with personal tastes, and we can relate to others of similar taste more than to those who are closer to us by some arbitrary measures.

That doesn't mean that a 12 year old girl's review is worthless, but it all goes back to what Dan Howdle and others are saying: the important thing is to have a reviewer who is knowledgeable and can express himself or herself, is preferably aware of his or her biases and provides enough detail for the reader to make an opinion. As someone looking at reviews, it's important to find a reviewer whom you understand. And yes, if the choice was between a 38 year old guy and a 12 year old girl who both like similar games and can express themselves equally well, then the 12 year old girl might have an edge over the 38 year old guy when it comes to addressing a 12 year old girl readership.

Posted:A year ago

#17

Login or register to post

Take part in the GamesIndustry community

Register now