As we continue the series of editorial features penned by the popular indie team at Introversion, the spotlight moves onto the testing and feedback process. Producer Byron Atkinson-Jones explains some of the challenges the company has faced in the past, and a few of the lessons learned.
Who Needs Testing?
Working for a small developer like Introversion has some fantastic benefits. The entrepreneurial vibe makes for a fantastically rich and creative environment, you have the opportunity to get involved in all aspects of running a business you wouldn’t usually get a chance to work in, and our flexible work ethos ensures a good work life/balance.
There are of course down sides too, and one of those is that we don’t have a massively large amount of resources to play with, which means that important activities like game testing are down to us and nobody else. This can have a major impact on the playability of the game because as anyone who's developed a game knows, being the developer makes you fundamentally too subjective and biased to be a good play-tester.
When you've taken an idea, nurtured its embryonic form, built it lovingly from scratch into your fully-fledged all-singing, all-dancing next IP, it can be almost impossible to see it from the naive perspective of the first-time player, and yet this is essential if you are to make a game that appeals and makes sense to your audience. Our experiences developing Darwinia+ for XBLA and working with Microsoft have taught us a great deal about the fine art of playtesting and we have come a long way from where we started.
Using Feedback Well
From the beginning, Introversion has always relied on in-house testing and help from a loyal and supportive group of beta testers, most of whom come from the Introversion forums. This approach - although limited - seemed to work well enough for Introversion in the past; since we made PC games it was easy to deliver preview code to third parties and particularly important, it was cheap.
So the first time we really got bitten from using this approach came as something of a nasty shock. It happened last year, during Multiwinia's development. With the PC version of Multiwinia close to release, we trekked down to the Future offices, as is becoming something of an IV tradition, to show preview copies of the game to the likes of PC Gamer.
Once we had got past the usual hassles of network setups and praying hard that the game would actually run at all, the preview seemed to go well - it wasn't until we were out of the offices that Chris revealed that the guys at PC Gamer had not liked the control scheme at all. This had devastated Chris and he then spent the next two weeks locked in a room coming up with a better control scheme based upon the feedback he got.
You Can't Please Everybody, But You Still Have to Try
It's a familiar problem in game development coming up with a gameplay mechanic that will appeal to many players rather than just a few. When we as developers sit down and design a control scheme we don't deliberately set out to design a scheme that is as obscure as possible, but rather one that makes sense. We then implement and test it ourselves and if something doesn't work then we go through the process again and again until we settle on something that does.
The key thing here is that the control scheme appears to work for us - up till this point nobody but the developers have had a chance to play with it. Since the controls tend to be put into place rather early on, the developer gets quickly acclimatised to his chosen control scheme and it's easy to become convinced that what we have now is actually a very good system and therefore best left alone.
This introverted approach to playtesting also backfired for us with Darwinia+ - just before we entered into the Code Complete phase of the project our account manager at Microsoft expressed some concerns over the control scheme we had in place. Our problem was one of coherence between Darwinia and Multiwinia and also the fact that our control scheme made use of context sensitivity which meant that controls had meanings based on the current action.
What had felt automatically intuitive to us was deemed to be confusing by Microsoft and they wanted to suggest alternatives schemes. Many months of iterations ensued before we finally settled on a control scheme that both parties were pleased with. Again, we have no real way of knowing if it will be universally liked but we are a lot more confident this time round because Microsoft had introduced us to the concept of usability labs.
Seeing is Believing
Usability lab testing is simple - you take a game, a few volunteers, stick them in a room and film them playing. You don't just get to see people playing the game; you also get to hear what their thoughts are on what they are seeing. Through this process we now have hours of footage of people playing Darwinia+, which one of our coders, Leander, was given the unenviable task of sitting and watching all the way through, taking notes of all of the comments and any areas the players had difficulty with.
One thing that became increasingly clear from watching the videos was that console gamers are not the same as PC gamers. There are a certain amount of assumptions that Introversion had made about who would be playing Darwinia+, and they were based around the hard-core PC-gaming Introversion fans who had played all our games before. We had unconsciously developed a particular developer mindset that naturally evolved out the years of making Introversion games and was suited to only that audience.
We quickly learnt, for example, that XBLA gamers are as likely to be novice gamers as hard-core gamers, principally because no-fuss digital distribution services such as XBLA appeal and cater very well to those with a minimum gaming experience. We had to rethink the theory that a player would instinctively know what they needed to do in-game rather than be explicitly told.
This is actually a lot harder than it sounds - it's far too easy to think that something is obvious in the game when the reality is that not all of the players will get it. It also had massive implications for us in terms of Darwinia+, affecting all levels of the game, from the tutorials, to each mode of the game in both Darwinia and Multiwinia, and has been a major factor behind the project's delay.
As a developer, one of the easiest traps to fall into is making a game that appeals and caters exclusively to your own particular tastes. You can probably get away with this more easily if you have a loyal fan base that have been following your games since the beginning because chances are you share a common interest.
However, if you are keen to progress as a developer and want to explore other audiences and game styles, then an understanding of this audience's expectations and assumptions is critical. We've learnt that this can only be done by opening your doors to the gaming public and encouraging a candid overview of the game as it’s in development.
For PC-centric developers, such as Introversion, keen to broaden their audience appeal and experience level, the move to console development can be a time-consuming, frustrating and painful one, but absolutely imperative if you are ever to grow and evolve.
Byron Atkinson-Jones is producer of Darwinia+ for Introversion.