GameDiscoverCo founder: "It's not selling out" to use metrics in development

Last month, Simon Carless ended his 16-year run with Game Developers Conference organizer Informa and its predecessor UBM. Yesterday, he announced his next endeavor, a research and consultancy firm called GameDiscoverCo.

As the name implies, GameDiscoverCo is a research and consultancy firm that seeks to help crack one of the biggest challenges facing developers today: game discoverability.

It's not running a live event like GDC in the middle of a pandemic, but game discoverability isn't exactly an easy task to tackle either. Still, it's something Carless believes he's prepared for. From his days as editor-in-chief of Gamasutra and Game Developer Magazine through chairing the Independent Games Festival, co-founding the Independent Games Summit track at GDC, and overseeing the entire conference, Carless has always had to keep abreast of how the industry is changing for creators.

Specific to the topic of discoverability itself, he points out to GamesIndustry.biz in a recent conversation that he's been running the Game Discoverability Now newsletter for over a year.

"It's definitely been the case that I've been enthusiastic about the space. I've always been excited about it, and the timing ended up being right to branch off and do my thing."

"It's not mercenary and it's not selling out to actually think about how many wishlists you need to launch..."

Carless believes that GameDiscoverCo can serve a key role for developers by introducing a more data-driven approach to development than is typically seen from the independent PC and console development scene. To do that, Carless knows he'll be working against a certain stigma around metrics among some independent developers.

"It's not mercenary and it's not selling out to actually think about how many wishlists you need to launch, and how you're going to get them and have a real practical plan for it," Carless says.

Specifically, he wants to separate the idea developers have of how metrics can help them from the trends they see regarding metrics impacting mobile game design.

"I think the mobile space is great for people in the mobile space, and that's where a lot of the money is being made," Carless says. "But because it's so metrics-based, the game designs tend toward, 'How do we get extra money from the player?' Obviously, the reason for that is not necessarily that the developers are being overly mercenary; they just need to acquire new players, and the way they do that often is via paid ads, and you can't compete for paid ads unless you have a game that monetizes very well.

"For me, mobile is all about the metrics, because you can't scale a mobile game without good metrics. Whereas obviously in the premium PC/console space, there's quite a lot we can learn from being geared with DLC, price elasticity and stuff like that. But historically, people have been used to the retail paradigm, which was you made a game, it was put on a shelf, and because there weren't many games on the shelf, you automatically sold like 50,000 copies. Now there are tens of thousands of games on the shelf, and it just requires a different approach."

The first steps of that approach involve tracking various metrics heading into release to get an idea of how much interest there is surrounding a game.

"How many Steam Store wishlists do you have?" Carless asks. "Presumably you have a Discord, how many active users do you have in that Discord? What's the amount of chatter on the internet? How many views have your trailers got?

"I freely admit it's not like metrics are the be all, end all. I often say they're indicative, and I think that's a really important word"

"All this type of stuff should be pretty indicative of how your game is going to sell. And I think many people will say, 'Well that was the first trailer; I have another trailer coming up' or 'Just before launch some really big streamers are going to pick up the game and it's going to be fine.' And that does happen sometimes... but I don't think you should be depending on that."

Carless emphasizes the importance of a strong launch, saying that fewer than 5% of titles significantly improve their sales trajectory after the first few weeks of release.

"I freely admit it's not like metrics are the be all, end all," Carless notes. "I often say they're indicative, and I think that's a really important word. A good example is I recently did a survey about Steam and how many sales you might have if you have a certain amount of reviews on Steam, and the range I got was anywhere between 20 and 60 times your number of reviews was your number of Steam sales. And that's quite a large difference."

That doesn't mean Steam user reviews are not a useful indicator of sales, he cautions.

"It just means you maybe need to have a range of data to base your expectations on, and you need to think a little bit about what kind of games are actually popular. Don't go the whole hog and have a game that's precisely tailored to a particular success that you don't even want to make yourself, but you should at least think about it carefully."

He also acknowledges that increasing these metrics don't always translate into increased sales, which he illustrated by pointing to the way developers frequently push people wishlist games before launch.

"There are now more pre-release opportunities to get your game out there via demos or features, and those end up racking up a lot of wishlists," Carless says. "And I think those wishlists are potentially not of the same quality. They're still worthwhile, but they're not of the same quality as the organic wishlists."

He adds, "That's why I say these metrics are indicative. Concentrating too much on just moving the numbers has led to some people going public about having like 18,000 wishlists at launch for various reasons and they only sold like 800 copies in their first week. How did that happen? And the answer is they weren't the right wishlists. And that gets into more complex questions."

"People just feel in their gut like a game is going to work, but there's additional information you can pull that will help you work that out"

It doesn't help that the best practices for discoverability can change quickly, and the abundance of people following them can essentially undermine their use. For example, Carless points to the Switch launch, where many developers who were on the platform early said it had been an amazing experience for them. Now he says there are 20 or 30 games released on the system every week and the advantage of being there has been thoroughly blunted.

As for what he sees coming around the bend, Carless points to the potential of subscription services like Game Pass Ultimate to reshape the discoverability discussion if the people offering them can realize their ambitions.

"If that really happens, it's going to be a massive change to discoverability," Carless says, saying it would be a multi-year transition for the industry if it comes to pass. "On things like Netflix, you still have to find the thing you want to watch on there. But the person who made the content has already been paid for it, so that's just a very different setup."

Regardless of what happens with discoverability going forward, Carless says GameDiscoveryCo's role is simply to help developers and publishers explore and understand it.

"Maybe they have a portfolio of games and they want some research-based input on whether they're in the right sectors, or whether they've got the right pre-launch hype," he says. "Maybe they've got a bigger problem with discoverability. But it's just adding knowledge and data to the gut feel we tend to use a lot of times when making games. People just feel in their gut like a game is going to work, but there's additional information you can pull that will help you work that out."