Close
Are you sure? Are you sure you want to report this comment? I understand, report it. Cancel

Fun Is Not A Metric

Fun Is Not A Metric

Tue 03 Dec 2013 8:27pm GMT / 3:27pm EST / 12:27pm PST
Design

Developers and publishers can't tweak their way to entertainment, but need to focus on fun

There's a danger lurking the game industry - the rise of the metric-driven game that aims to maximize monetization without consideration for how fun the game may be. Those sorts of games may make some money in the short term (especially if done quickly, usually by copying an existing type of game) but they don't have a lasting value. Worse, such games tend to devalue the whole idea of game playing on a platform, and can lead to an exodus of players. That may be part of the reason why social games dropped in popularity, though the rise of mobile games accounts for the majority of the switch. Mobile games, though, are certainly suffering from this issue, and games on other platforms could be too. How did we get to this point, and what can be done about it?

The rapid expansion of the game industry into new platforms, new markets, new business models and new demographics over the past ten years has resulted in a Cambrian explosion of games. This growth sparked the creation of many new game companies, as both developers and investors have sought to take advantage of the massive growth. The rush to create and grow new companies has inadvertently resulted in too many games and companies where the focus is on metrics, rather than fun.

It's a terrific advantage for game designers to get large amounts of reliable data about how people are playing their games. In the past, the only information designers received was anecdotal, through reviews or the occasional letter. Now mobile and social games get incredible amounts of data from every player, and PC and console games are beginning to do the same. Designers can see exactly where players have difficulty, or it's too easy, and tweak the game accordingly. Company execs have also seen how the game can be tweaked to improve revenue, watching the effect of adjustments to the design and how it alters income.

"Investors in game startups should be looking for the fun in what developers are proposing, not for just for their engineering talent or advanced analytic chops"

The danger that's manifested all too often is that the metrics end up driving the development of the game. There are game companies whose raison d'Ítre seems to be maximizing revenue by tweaking the psychology of game players. Sure, companies usually are trying to maximize revenue, but when you're making games it seems that creating a fun game should be right up there among the top priorities. Shouldn't it? If it's not, shouldn't you be in some other business?

Eventually gamers get tired of being played instead of playing. While free-to-play games make it easy for new players to enter, the vast array of free-to-play games available make it easy for people to switch to something else. We've seen this with many mobile games that appear on the charts, blaze brightly for a while, then fade. Former EA CEO John Riccitiello pointed out that we have yet to see a successful sequel to a mobile game, or a mobile game franchise that has had anywhere near the longevity or revenue of the best console or computer game franchises.

Console games are beginning to find out just how much the console gaming audience will accept microtransactions, with games like Forza 5 where you can pay for new cars. Most console games now have regular doses of DLC arriving, and we may see that DLC available in smaller and smaller pieces. How well will this work in the long term, and is there a point at which there's too much DLC for a game? We don't really know the answers yet. We need to be careful how we find out those answers, though.

There are some wildly successful games using the free-to-play model on the PC, like World of Tanks and League of Legends. Those games are very different, but what they have in common is a focus on making the game fun and continuing to improve the fun factor. Wargaming CEO Victor Kislyi is proud of the fact that half of the company's employees deal directly with customers, working in support or community roles. Kislyi does look at metrics, but stays away from constantly tweaking deals in order to squeeze a little more money out of users each month. He feels the game will do better in the long run by focusing on making it better and more fun, which will keep users engaged for the long term.

Adding fun to a game is extraordinarily difficult. It's not a dial you can easily adjust to add 23 percent more enjoyment. It's not a metric you can read directly, though you can infer the amount of fun people are having to some extent by the depth of their engagement with the game and the number of hours put in. Finding out exactly what it is that players enjoy about a game really requires conversation, and engaging with gamers on community forums, message boards and social media. Getting that information back to game designers is critical, but there are no guarantees. Fun is elusive, and magical, and sometimes you just have to get lucky to find it.

Games are now more than ever a long-term proposition, at least if you want the effort put into the game to really produce a substantial profit. Big publishers increasingly want every game they produce to be something that attracts and retains a large audience for a long time, with plenty of opportunities to sell things to the gamers and spin off other products. That can only happen when a game is really fun. All the marketing and brand extensions and DLC in the world won't get an audience to stick around in the kind of numbers big publishers need. The game has to be great fun, with prospects of more fun to come in the future, to create the kind of audience publishers crave.

That fun factor is key for smaller developers, too. Yes, it's hard to find an audience these days, and getting harder. If your game is fun, and you are patient, you can find and grow your audience over time through recommendations. (The caveat here is that many small developers can't afford to wait a year or two for a substantial audience to develop, so this must be taken into account in the planning phases.)

Game designers, developers and publishers can only build lasting, long-term value with a fun game, and that has to be the core value for a game company to be around for the long term. Investors in game startups should be looking for the fun in what developers are proposing, not for just for their engineering talent or advanced analytic chops. Do that, and the rising cost of user acquisition won't bother you. Let's all have more fun with gaming, and the rapid growth of the game industry will continue.

Republished from our sister site the [a]list daily. For more, read the [a]list daily and subscribe to the newsletter to get the latest in game and entertainment marketing news, cool videos, incisive opinions, exclusive interviews and industry data.

21 Comments

Paul Johnson
Managing Director / Lead code monkey

810 1,013 1.3
Popular Comment
People look at me like I just shat on their desk when I tell them we don't take metrics.

Q: "But how can you know what your players are doing"
Me: "Because I know how to make games and all the user reviews are 4 or 5 stars"
Q: "Yeah, but how do you know about retention"
Me: "I'm quite regular thank you"
Q: "You know what I mean"
Me: "Not really. I make games for a living. If people buy them and enjoy them for a while, I'm done"

Edited 1 times. Last edit by Paul Johnson on 3rd December 2013 10:36pm

Posted:8 months ago

#1

Bruce Everiss
Marketing Consultant

1,692 594 0.4
This is a rare instance where I disagree with Paul.

Yes, games should not be metrics driven. Creativity, skill and experience are what make new and compelling entertainment experiences.
BUT. Metrics give an immensely powerful insight into player behaviour. You can see how your game is actually being used by the real world people who are playing it. This allows you to polish, to remove problems that you would otherwise be unaware of, to make a good game into a great game.
The power of metrics is so great that I cannot believe or understand anyone not taking advantage of them. They are an integral part of games as a service and of looking after your customers properly.

Posted:8 months ago

#2

Steve Peterson
West Coast Editor

108 73 0.7
Metrics are a wonderful tool, but should not be a substitute for creativity. Designers should use metrics, not the other way around.

Posted:8 months ago

#3

Paul Johnson
Managing Director / Lead code monkey

810 1,013 1.3
We run a forum to get actual feedback. You can read far too much wrong stuff very easily out of metrics I think.

We seriously considered doing it, given it seems to be all the rage, but honestly couldn't come up with a single real world scenario that would allow us to improve the experience/income/anything that we didn't already have access to. And without abusing people's data plans.

But I'm not close minded on this. If there is some big secret I've not discovered, I'd love to hear about it please.

EDIT: For example, this:
...This allows you to remove problems that you would otherwise be unaware of...
How exactly? Let's say that 10% of downloaders quit on tutorial 4 and I see there's nothing buggy or boring about it. Now what?

Edited 1 times. Last edit by Paul Johnson on 3rd December 2013 11:44pm

Posted:8 months ago

#4

Allen Bevans
Associate Games Analyst

1 0 0.0
Let's say that 10% of downloaders quit on tutorial 4 and I see there's nothing buggy or boring about it. Now what?
You use player research like 1-on-1 playtest / interview sessions to find out why those downloaders are quitting. Even though you don't see anything buggy or boring about tutorial 4, 10% of your potential player base is leaving for what they consider very valid reasons. It is likely that given the right insight, you can do something to address their concerns and stem the drop off.

Posted:8 months ago

#5

Nuttachai Tipprasert
Programmer

79 60 0.8
10% of your potential player base is leaving for what they consider very valid reasons.
Or, perhaps those 10% were not your target players to begin with?

I understand what you mean; I'm working in social games company as well. But Paul has valid points here and I agreed with him. If 90% of your players doesn't find anything boring or wrong about the tutorial or whichever game section that 10% player dropped off, that means there's nothing wrong with that section and those 10% are just not the target audiences the game is aiming at.

In most cases, compromise need to be made in order to keep those 10%, who might not be your target audience, playing your games and that might drive away the other 90% who already satisfied with the games.

I think metric is a more suitable tool for marketing than game design. Like the article stated you cannot gauge the fun factor of the games. If it's as easy as that, every fortune investors should become legend game designers by now but that's not happening.

Posted:8 months ago

#6

Paul Johnson
Managing Director / Lead code monkey

810 1,013 1.3
It is likely that given the right insight, you can do something to address their concerns and stem the drop off.
Yes, but HOW exactly?

Hubris aside, no developer should push stuff out they are not happy with, so it should be a given that I think tutorial 4 is just fine as it is. But my metrics are telling me there's a problem. So the next step is to randomly change something I know nothing about. That could just as easily result in 20% of players leaving instead.

The word "insight" that you used is the crux of this. Without it, all the numbers are pointless.

Posted:8 months ago

#7

Bruce Everiss
Marketing Consultant

1,692 594 0.4
Hi Paul.
At Kwalee we knew how many people were arriving at our game page on the App Store. We also knew how many people got to the point of making a first move in the game. And we were losing about 9 out of 10.
So we put maximum metrics on the "funnel" and observed what was happening. Where the metrics said that we were losing people was not obvious. We were able to clean things up according to the metrics and the number of people getting to the first move went up several fold.

I cannot believe that anyone wouldn't want to know this stuff. It is telling you what the customer is doing with your game.

Posted:8 months ago

#8

Paul Johnson
Managing Director / Lead code monkey

810 1,013 1.3
Sounds like a win Bruce, but you missed off the "how" part.

Posted:8 months ago

#9

Alfonso Sexto
Lead Tester

786 595 0.8
@Bruce & @Paul

Talking about rare instances, This is one of those cases in which I agree with Bruce.

Paul is right when he said that the important thing is that people enjoys your game, have fun, a good experience and a good memory in general. You build reputation with that and people will remember you next time you release a new tittle (hardcore gamers at least, for casuals you can just add some marketing to refresh their memory ;) )

But the thing is: how do you know that those people are having fun? how do you know what part of your game they like and which one they skip/stay away from?. You need metrics for that because of something very simple that applies to every industry: you can please everyone and you can't listen to every opinion independently.
I cannot believe that anyone wouldn't want to know this stuff. It is telling you what the customer is doing with your game.
Exactly! This is specially important in F2P products where the initial drop-rate is always very high and the retention level is always a top priority. Like Bruce pointed before: a loose of about 9 out of 10; a very high loose rate that at the same time is quite common
Yes, but HOW exactly?
GTA Online has a popularity % in every activity, so even the players online can see what people does the most and which maps they usually play into. OR you can build a comunity in a forum like Blizzard does or like we do here in Ubisoft.

Posted:8 months ago

#10

Andrew Watson
Programmer

92 200 2.2
How exactly? Let's say that 10% of downloaders quit on tutorial 4 and I see there's nothing buggy or boring about it. Now what?
How about, oh, I don't know, asking them? Looking at things they say about the game? Digging around in forums and social sites for people talking about your game and find consensuses on parts they don't like?

Posted:8 months ago

#11
Metric is just a way of saying hidden poll. Doesn't matter how you do it, it might be handy as a tool but it's just a tool, I wouldn't found a religion of games development based on it, because games dev is not a exact formulae. Luck, fun and opportunity cannot be tweaked with metrics

Posted:8 months ago

#12

Paolo Giunti
Narrative Designer

35 68 1.9
Metrics are important, but, as Steve said, they're just a tool to us that should not replace creativity.
You need to find the right balance between the two.

Just giving a couple examples from personal experience:
A couple years ago i was working in an independent studio (the kind of studio that strongly favors creativity. Officially i was QA but, as often is in small studios, you end up ignoring your job description and do whatever you can do to make yourself useful, so i was also the metrics guy in there.
I recall about one mystery/dark/supernatural-themed RPG project (which i originally endorsed and backup up with my data) that was later turned into an action-adventure, more hack 'n slash, kind of game. That's where I voiced for changing the setting into something more colorful. Dark gritty and mysterious for an RPG filled with puzzle/investigative quests: it works awesomely. The same on a fast-paced action game? The data i had really had me worry.
Then there were some guidelines, such as: "make controls as intuitive as you can. XBLA audience wants 'fun here and now'. Avoid long intros and tutorials."
To this extent, i think putting an eye on metrics can be healthy to the game.

Now i happen to work for a company where metrics are everything and I confess i don't see much innovation in the games. They all go for the same winning MMO recipe with a slightly different flavor. All these games are pretty much instant cash machines, but, i tell you what, i don't expect any of them to actually last more than a couple years.

Posted:8 months ago

#13

Paul Johnson
Managing Director / Lead code monkey

810 1,013 1.3
How about, oh, I don't know, asking them? Looking at things they say about the game? Digging around in forums and social sites for people talking about your game and find consensuses on parts they don't like?
You mean like when I stated we use a forum to do exactly that?

Posted:8 months ago

#14
@Paul *Applause* for you sir. Mobile experts love to tell us how we devs now have to learn marketing, PR, data analysis and metrics based design or else go home. How about those same experts learn about compelling worlds, playability, game flow and why players love gaming without metrics telling them what to fix? They might find they don't lose those players they spend million$ trying to retain.

Posted:8 months ago

#15

John Foster
Senior Designer

3 3 1.0
@Paul,
Hi Paul. I don't think anyone would argue that metrics tell you how to fix a problem, that is of course where we rely on our experience as developers. Metrics can help you identify that a problem exists however. Forums are very useful for getting certain types of information from players, but they are for one prone to selection bias (some people use forums, some don't, so you only hear from the players that do). Forums as a source also require that players be aware of the problem themselves in order to be able to tell you about it (for some issues, that is fine as the issue is clear to all users that encounter it, but for others it is not). Even after identifying that a problem exists, players then need to understand the issue well enough to describe the actual root problem (I suspect that your staff, given sufficient data, could determine the root cause of negative player behaviour more effectively in many cases).

We don't have to choose one over the other. I agree with Barry that we don't have to rely on metrics to tell us how to create compelling worlds, or good game flow. Neither do we only have to rely on forums as our only source of information about the player experience. I tend to think metrics are just a name for a certain type of information, it is just user data in a raw form. So long as we can make something useful of that raw data, why would we not want to have access to it? Simply having information doesn't dictate what we do with it.

Edited 1 times. Last edit by John Foster on 4th December 2013 4:58pm

Posted:8 months ago

#16
I think metrics can be useful, but dont go spending an arm leg and the whole house for em to necessitate its data collection. For that you can dial NSA 101!

Posted:8 months ago

#17
The trouble with this is that you are talking about game metrics. Let's say you had a poor retention problem. The game played well in the first session as far the developer was concerned. BUT the developer no longer represents the average player - and it is very difficult for developers to imagine the experience each type of player is having in the game. That's where analytics comes in. Analytics can get you to the top of the mountain but can;t find the right mountain. It cant save flawed games but it can make good games great. For example, by recogniizing player competence early in the first session you can give novices and experts the best experience for them. This is the power of analytics. You can't do this from forums. Know and respond to your players and you will build successful and profitable games.

Posted:8 months ago

#18

Gavin Price
Design Consultant

8 3 0.4
I think metrics can be most useful to gauge usability and help spot broken functionality, whereas creativity and forums (or similar customer engagement) are more useful when creating fun gameplay where you can gauge subjective discussion with/against your own feelings. Bit of both in the right context and hopefully a great game come the end of it.

Posted:8 months ago

#19

Paul Johnson
Managing Director / Lead code monkey

810 1,013 1.3
#20

Fernando
UI Designer

1 0 0.0
Let's get the official statement out of the way first:
I guess it's a valid approach for some companies. To me that's something we have to accept and to a certain point, embrace if we want our 'industry' to be able to grow: the fact that there is not one absolute truth on how to do things, and the variety of end products this concept generates. Be this metric-driven or any other way. However, this should work both ways (yes, data-driven preachers, I'm talking to you!)

On a more personal approach, to me all this metric-driven frenzy feels more like a lot of people interested in making a safe buck based on these 'new shiny tracing tools' without understanding that like in any art medium or process, risk (be it financial, artistic, content-wise, you name it) is not something you simply tick off of a list; it's an important aspect of creation.
I don't think we should treat creation (of any type) as some sort of dark magical process, but if it was a question of extremes, the data-driven extreme seems to take out all what makes this medium compelling.

Edited 1 times. Last edit by Fernando on 6th December 2013 9:26am

Posted:8 months ago

#21

Login or register to post

Take part in the GamesIndustry community

Register now