Do You Speak Metrics?
Playmetrix breaks down the new language of game analytics
In this guest blog for GamesIndustry.biz, Justin Johnson, chief technical officer of game analytics firm Playmetrix,breaks down the new language of game metrics.
Gaming metrics is still a young field of analysis with a multitude of approaches taken by service providers and developers/publishers alike. Even so, the world of gaming metrics comes with its own panoply of terms, definitions and acronyms. Many of these terms have roots in internet marketing although some are from further removed fields - for example K-Factor is a term borrowed from Virology.
In this article I'll describe some of the more common terms and acronyms in use along with a description that will shed light on what they are and what they mean.
An event is a discreet record of activity logged by the game and usually sent to a remote analytics system, in our case the Playmetrix collector server network. Examples of an event could be when a player starts or finishes a game (session events), upgrades inventory items, gets a reward or enters a particular game area or level. A metric analysis system can then digest these events - usually in their millions - and produce meaningful time series visual data in the form of charts and tabled reports.
Uniques are a useful term for talking about events when it only matters that the event happens at all - the frequency of the event is not significant. For example, if we ask the question 'how many players turned up today?' it doesn't matter whether a particular player played once or several times. We'll just treat the fact that they turned up at least once as a unique count. Another example might be 'how many players played the haunted house level this week?' A particular player may have played that level several times during the week but we only want to count that once.
DAU is an acronym for Daily Active Users. It is the unique number of players that played your game in a 24 hour period. It doesn't matter if a particular player plays your game several times in this period - their attendance is counted only once. A graphed visualisation of this metric gives a good indication of daily activity. At Playmetrix we use a variation where the 'U' stands for 'Uniques' rather than 'Users' and we generate this metric for all known events.
WAU is an acronym for Weekly Active Users. It's calculated in exactly the same way as the DAU except the time period is now 7 days rather than 24 hours. Interestingly, the metric is not as commonly used as DAU and MAU, although obtaining visibility on a 7 day period is useful in practice.
MAU is an acronym for Monthly Active Users. This metric also is calculated in the same way as DAU and WAU except that the period is now 30 days. A 30 day period is a good mid-term range for viewing metric data and making decisions based on subsequent analysis. With this metric you're now recording whether a player attends at least once in the month. As with DAU, at Playmetrix we use a variation where the 'U' stands for 'Uniques' rather than Users and we generate this metric for all known events. That means we can produce an MAU graph for a specific game event rather than player attendance alone.
This is a ratio calculated by dividing the DAU by the MAU. Converted to a percentage, it answers the question 'what percentage of my monthly players turn up each day?' For example, given a MAU of 600,000 and a DAU of 30,000 gives 0.05, that's 5 per cent of the total monthly players turning up each day. The target values for this metric can be fairly subjective but top Facebook games look to hit 25 per cent and higher. It's been cited that reaching 15 per cent is one of the indicators that the game is 'sticky', has a low enough churn rate and is viable for increasing advertising spend. At 15 per cent plus, the game will hold onto players long enough to ensure healthy growth with low attrition.
The Average Revenue Per User is calculated by dividing total revenue by the total number of players. Seems simple enough but it becomes interesting when you have a section of your player base that doesn't pay anything - for example the players joining a freemium game, or those that are on a free trial period. The ARPU gives you an idea of the revenue generated by your entire player base, apportioned to each player, and may be reported using time periods. For example, what was our ARPU last month? What's our ARPU so far this year?
We may be interested in finding out the revenue generated from each paying player and this is where the Average Revenue Per Paying User metric comes into play. This is calculated by dividing total revenues by the number of paying players. Unlike the ARPU calculation, we ignore the free and trial players in this calculation as they don't contribute directly to revenue.
Lifetime value of a player refers to the amount of revenue attributed to an on-going relationship with a player. That is, given a single player, how much have they spent with us from the time they first played our game? A prime use of this metric is in calculating what our acquisition costs are in relation to it. For example, if the LTV is significantly lower than our customer acquisition cost then we have a problem! In this case we're not earning enough revenue from customers to account for the acquisition spend in obtaining them in the first place. The LTV can be calculated as: Monthly ARPU x Lifetime of player in months. We can integrate K-Factor using a first order approximation to give the LNV (Lifetime Network Value) which takes into account the other players that a player may invite via viral channels: 1 / (1-k) x Monthly ARPU x Lifetime of player in months where 'k' is the K-Factor (see below).
K-Factor is a measure of virality for applications/games that utilise viral methods for player acquisition. Given a customer base (C), number of invites sent (S) and % of invites accepted (A), K is calculated as (S x A) / C. If K is lower than 1 then your player acquisition is decreasing exponentially, a value of 1 implies steady state and K of greater than 1 indicates exponential player acquisition.
Duration refers to a time between game events - often 'start' and 'end' event types. It is used to answer questions of the nature 'how long did the player do x for?' For example, 'how long did it take the player to complete level 5?' or 'how long did it take for the player to get the reward after purchasing an item?' Measuring player behavioural durations indicates whether players are acting within the timelines that we expect them to.
A session starts when the player starts to play the game and finishes when they decide to do something else. Sessions tell you how frequently a player plays your game (three times a day, six times a month etc) and by measuring the duration of sessions we can work out the average time that they spend playing the game in any given period. Sessions give a very useful high level metric for game activity and engagement. The end of session is usually marked by a period of event sending inactivity. For example, if a game hasn't sent any events for the last 30 minutes then we can assume that the session has ended. This has the effect of quantizing session durations. Only if the game has a persistent socket connection to the metric servers is it possible to detect precise session end times upon disconnect but as most metric systems are REST based, that is they send simple web requests to log events, this is usually not the case.
One of the main reasons for using metrics is to refine your game to deliver better entertainment to players. As an entertainment experience your game should have high level objectives that you wish the player to fulfill. The purpose or objectives of the game can be design related ('we want our players to finish all levels/build a city/make friends') and they can also be financial ('we want the player to buy virtual goods after finishing level 1' or 'we want the player to subscribe after a free trial'). Either way, you're wanting your players to convert from one state to another, from inexperienced to experienced, from free to paying etc. That's a conversion.
A funnel is a multi-stepped filter where a given number of units (often a unit is a player in game metrics) start out at the first step and are then filtered away in subsequent steps. Each step is illustrated in the funnel graph along with the number of players that make it to each step. The number of players associated with each step can also be shown as a percentage - either a percentage of the whole or the percentage of players from the last funnel step. Each funnel step can be specified with appropriate filtering logic leading to a final conversion figure at the last step. For example, we may have 543,343 players at the top of the funnel but only 12,000 are present at the last step yielding a 2.2per cent conversion. A funnel may be set up to show conversion for financial game transaction event or they may be set up to show pure game play conversions. For example, 'how many people got to level 10 (1st Step), then how many bought the big gun (2nd Step) and then finally how many went on to kill the end of level boss (3rd Step).
A cohort is defined as a group of players who share common attributes. Cohort analysis provides the ability to track the churn/attrition rate of these players. Cohort charts are often represented as a two dimensional matrix showing a time period split into smaller division along each axis. The cells of the matrix contain the player count. For example, a row of the Cohort chart for a six week period, split into weeks, will show the player counts at week one through week six. The player count will decrease in each column cell along the row showing our attrition rate from the count at week one. The next row will show week two to week six, the next will show one week three to week six and so on. A Cohort chart gives detailed figures on player attrition although in Playmetrix we can calculate Cohort analysis for any given custom event, not just attendance. This allows the measurement of game feature adoption and repeat play.
Sometimes called multi-variant testing or A/B testing (although we might be splitting on A,B,C and D!), Split Testing is a feature optimising technique. It's implemented by randomly providing the player with a differing version of a particular feature and then measuring the effectiveness of that version. It may be purely cosmetic - for example one kind of button style against another - or it may be a split on player interface features or even deeper game mechanics. The important thing is that the effectiveness of each variant is measured and, at some point in the future, based on effectiveness, a split can be chosen as the surviving feature/version. Applied effectively, this technique allows a game or app to be continuously refined according to majority player tastes and preferences.