Everyone's a Critic
Metacritic's Marc Doyle on the career score controversy, responsible reviewing and credit where it's due
From the start our mission hasn't changed. It's a consumer-focused site. How can we make the job of picking a game easier and a more educated process for a gamer? So let's cut through all the PR stuff and the marketing and the advertisements. What do the top critics think about this game, relative to another game? So if you see one game scores 85 and another scores 62, at least based on what the critics think you should go for the 85 game. It's as simple as that. It's a tool among many tools.
If you happen to like that type of game, go for it! I've certainly bought games that have had red scores, I've loved movies that have had horrible scores. It's just a tool for me to know what the critics think.
So we launched around 2001, it was probably around 2003 - 2004 when certain publishers really started taking notice of the metascores. And you know, my thought was: "This is interesting, because when we jumped into the field, and when Gamerankings.com jumped into the field, before that it was always just sales and they could put out sequel after sequel in certain big franchises not worrying about the quality and they were going to do that forever. And I thought: "OK, we're going to jump in here and try and help people."
Did I envisage the day that the industry would take these numbers so seriously? Not at all. In a way I find it a good thing, in that at least they're interested in quality. It's not necessarily for the sales of this particular game - it seems like there's concern with franchises and their licenses. Like if they put out a really poor Lara Croft game, to use an example, it might still sell like crazy but people are going to learn about it, know about it, and then the sequel's not going to sell as well. The value of the franchise won't be as high. I think that's partly the way they view it.
Reviewers will say, "here's what the top end of our scale means and once you go below this it's all kind of the same," but to me that's not responsible.
People certainly contact us, if I'm missing a score or they want to understand how we convert scores from a five star system. But I just try to be as fair and consistent as possible. It's been me from the start, so the philosophy behind how we pick sites and take scores hasn't changed.
I think it comes down to the consumer. I think a movie goer will still gladly go out and see movies that are critically panned, so they'll go see Transformers 2 en masse knowing it had a big fat red score on Metacritic. I'm the same way, it's a two hour commitment, it's $10. They'll go for it. But that's versus a game which is expensive and is a big time commitment and if people see a game is not a good game according to the critics they're not as likely to jump into that.
There's certainly a bit more inflation on the game side of things. If you see something in the 40s on the film side of our site, which is still a yellow/mixed score, then it's still fair game. But if it's sub-50 on the game side most people would describe it on their site, according to their review policies, as a very poor game.
But a lot of people will say to me, "Well, nobody ever scores really above 9, really below 7" but it isn't true. I could point to so many games where every review on the page is red. I think part of the issue is that movie critics, the big ones - the ones that are paid and do a really good job on it - they're reviewing everything that comes out that week. They'll review the crap.
Whereas the trend now with video games is that in contrast to 10 years ago they aren't reviewing the crap as much. They haven't developed the bottom end of their scale as much and I think that's a problem.
I tell as many people as possible about this: "Start reviewing the shovelware, tell me what a 2/10 means for you, tell me what a 3/10 means." So a lot of people will say, "here's what the top end of our scale means and once you go below this it's all kind of the same," but to me that's not responsible. I want you to be as precise with the bad as you are precise with the good.
I think that's where these veteran film critics are doing better than the veteran game critics. They have a tighter control and they have a greater understanding of the precision of the low end of the scale. Many games journalists also do this well, but I think it's an area where there can be general improvement.
For games we don't cite the individual journalist's name - we just have the publication listed. With movies they have a more consistent staff, they don't use as many freelancers. We do have pages dedicated to those people and those entities but it's simply: "Here is their average score, their highest score and their lowest score", but we're not scoring them personally. We'll do a basic analysis, but we're not evaluating them per se. Just how they score relative to other critics.
Marc Doyle is co-founder and games editor of Metacritic.com. Interview by David Jenkins.