Butterfly Effect

BioWare's Adrien Cho discusses the realities of managing the workload on the Mass Effect franchise

BioWare's Mass Effect 2 has quickly become one of the most acclaimed titles of the current generation, with the recent release achieving very high Metacritic scores of 96 per cent and big sales for publisher Electronic Arts.

Here, in an exclusive interview for conducted at last month's Nordic Game, Adrien Cho, BioWare's lead technical artist, talks about the realities of working on a AAA franchise, managing a constant workload while avoiding turning it into a production line, and opening up company secrets to outside help in order to create a better game.

Q: Can you begin by talking us through your position at BioWare and how many staff you manage?

Adrien Cho: I was lead technical artist on the original Mass Effect and I transitioned to an art production manager role, and I now work with the art director really closely to ensure the art gets made. So the art director gets the fun job of making sure the aliens are the right colour, and I have to get into the nitty gritty and make sure it all comes together. I help manage the teams and make sure work is done to deadline so we can move onto the next asset. It's a production line management. In the technical artwork role it was good because you could tag certain problems coming down the pipeline. This opens up a lot more so I can move resources around and not only spot the problems but now I can get artists to move around and prevent problems from happening. The scheduling aspect obviously makes everything a lot more efficient and it means we have less crunch and get the game done on time.

We have about six character artists, two concept arts, two VFX artists. Then on and off - because what we produce is tied into other teams - we have a team of two or three system designers that we work with closely, we have two animators, and it fluctuates. And then the external teams can range from between 30-50 people. If that happened all at once it would be mind-blowing, but the nice thing is that if we stage that and arrange things to take care of the most important things, and it's much more manageable.

Q: A lot of work involves outsourcing, which still has these bad perceptions cheap labour, farming out work to different countries rather than supporting your local industry...

Adrien Cho: It's the realities of making big games now. Even the smaller games. Coming from the art point of view it's important because you it's still viewed as a taboo, or a dirty word, because everyone's worried that you're sending out jobs from Canada, the US, the UK to places like India. That's the general idea. But the reality is when we talk about outsourcing we're talking about sending work back to North America or those same locations.

It's simply about understanding what your team can do in the amount of time they have. It's about changing that definition and stigma and about really getting the job done, getting the right help at the right time and how can you get the most out of that help. Sometimes from a project point of view people think that if they put money into a problem they can get it to disappear. But no, it's the same thing as working with an internal team. There really is no difference between working externally or internally. There are certain realities within that keeping things cost-efficient, understanding where your roles are but with an external team, even though the role is the same as an internal team, from a client point of view it's different because I drive the money. It gives me a different advantage because I'm paying for this product I have a lot more power dealing with external teams than with my internal teams.

Q: Does that make you more cautious because you're in control of the wallet?

Adrien Cho: You have to be more careful because you have to justify how you're spending it and you also have to consider do you even need to outsource it at all? That's a very key thing because if you send out the best stuff, you don't want your internal artists doing work they won't enjoy, and you don't want the internal guys cleaning up somebody's else's problems either. The way I approach it is I give my internal teams the best stuff that they want to work on. But the reality is there's too much work for the internal guys to work on and that's one of the reason why we have to outsource.

Another is expertise and understanding the capabilities of the guys in-house. It's going to vary from project to project but I look at the smaller teams in the studio and they're very focused on one thing and economically it does not make sense for them to hire in a compositor. Instead of having an half-assed compositor or get someone internally to learn that, why not just send it out? Maybe you pay more but it's because you don't have the in-house experience. When we're working with our external partners we're always thinking, how can they compliment our external team? It's really just working towards the end goal of putting the best product out there.

Q: What are the main priorities in outsourcing work for BioWare?

Adrien Cho: I always use that analogy of the mechanic where you take the car in and they always tell you the three things price, time and quality. You're only allowed two of them. It's a similar thing. Price from a project point of view is very important, quality for us has to be top-notch and with time I find that if you plan everything out properly you can get inexpensive work done to a good quality. If you don't have enough time you have to think how can you mitigate the risk? You may have to pay a bit more or the quality may go down.

Q: So how did you ensure those goals on a project like Mass Effect 2?

Adrien Cho: One of things that we worked on for the game itself was really not to have to reinvent the wheel. We invested a lot of time with the first game three and a half to four years to finish Mass Effect and it pays off because we've built this great foundation, from a technology point of view. You create this datum line that everyone understands, and up to that point, until we finish the game, that line always shifted. Because you're defining what does combat mean in the Mass Effect world? What does animation mean in a Mass Effect sense? What about buildings, vehicles, aliens? When you finish that first game a lot of answers are set now because this is how we did it. So we can then go back and re-evaluate those. We had planet exploration in the first game but it certainly could be better. So we changed it for the sequel.

Generally you have this foundation, so the sequel was just saying what are the concentrated areas that we focused on, what are the areas that we did well on an we can let them be? Even before we shipped the first game we knew the areas that we wanted to concentrate on and we could really focus on that right away. How does that work with external teams? It was the same in that we've built up a good working relationship with some of external partners and with the second time around it was really looking at where can we get the major gains from with the least amount of effort but unblock the biggest issues. So for example we sent out a lot of props for high-res modelling. High resolution modelling is very time consuming and costly internally, and it's not as much fun for our guys. Given that chance, an artist would rather work on a high-res gun than a high-res space radiator. But it's still a task that needs to be done. So these more mundane tasks that we have to flesh out the world we can send them out to external teams. We looked at the inefficiencies from the first one and said where can we improve on those?

Q: What are the important lessons from working so closely with an external team, and can you still reach that same quality threshold?

Adrien Cho: One of the very important lessons from that was how ever much you give out to your external teams is what you're going to get back in. It sounds dead simple to the point of dumb, but if a product came back that was not up to quality par, after looking at it maybe we weren't providing them with enough information. That's one of the key things we learnt you can't expect them to succeed if you do not invest time into them.

On Mass Effect 2 we spent a lot of time focusing on those aspects and seeing how can we ensure the communication channels are really tight, how can we improve the quality of the art that went out? And we also tried to catch all the really big things early on. The whole philosophy is how can we help them succeed rather than shove this on them and wait for it to come back. The basic way of looking at it is here's a contract, here's some money and bring us back our assets. But the other approach is how can we work together? And that pays off because we have a huge initial time investment but then over time you can walk away hands free. And if you get that machine running they will turn out the work. We were able to break things down.

Looking back we saw [external partners] were making extra textures that we didn't really need or use and therefore the assets were less efficient in game. So we said okay, let's make a standardised shader and by creating this repeatable set of actions for them we set a certain expectations from them and they get used to making it and they're able to make it 200 times. So long as the base pipeline is really efficient we found out that we're able get far more high-quality assets out, they're spending less time on them, and we can move much more quickly. Internally we might be spending between five and ten minutes per asset. But if you do that 200 times it adds up really quickly. So it's taking those small gains and sending it back out. As a whole picture we're able to look at small areas that we're able to get maximum gain from and tighten up on the areas that need working on.

Q: How do you keep consistency on a project when you're farming out work to external partners?

Adrien Cho: We shared a lot. Some of the principles sound absolutely silly but once we see the benefits of it it becomes obvious. One of the things that we didn't do with the first game was we didn't share tools and engine. We were secretive and you have to be careful of brain drain as well because you're training up and your knowledge base is sometimes what keeps you ahead of your competitors. On the other hand, if you're sending out work and it's not meeting your internal quality you don't want that discrepancy.

So for Mass Effect 2 we decided that for trusted vendors it was very important to be as transparent as possible and that philosophy actually worked out very well. We were actually quite surprised that the more we shared, the better the work that we got back. The realisation came when there was a huge ramp-up time and we were working with a studio called Blur, who do a lot of high-end CG work for trailers, and we gave them a lot of access to all our high-res models and all the information we had and worked with them very closely. The interesting thing that happened was artists are in a very small circle and our artists new their artists so we ended up learning from them. So we looked at some of the changes and were able to put them back into our work. It was a very symbiotic relationship were the philosophy was very much don't treat them as a bastard child. Even if they are only working with your for two or six months, treat them as you would like to succeed.

Q: Was there initial reluctance to being so open about your technology and processes with external teams?

Adrien Cho: That's something we learned from the first game. The second time around we looked at the risk factors. We still kept a lot of the key things in house, but we were able to send out a certain type of work and there was little risk that someone would steal that and make their own Mass Effect. The benefit was ten-fold back to us, you gain so much back. We worked with a Scottish outfit called Axis Animation too, and we were really happy with their work. They represent a new studio that we didn't work with before but we were happy to be transparent with them and they stepped up to the plate and did an amazing job. They were able to bring in interesting things. In the end it opened our eyes because it's a very fast-moving business world and your level of engagement can be very short sometimes. So even if your company has a short contract you want to get the best out them in a short time. And if we go back for future work the people might not be there. You have to make the most of what you have.

Q: You said Mass Effect 2 has come around quicker, and it's set you up nicely for Mass Effect 3 and the DLC in between, but how do you avoid that turning into a churn, slogging through the work to meet new releases?

Adrien Cho: I think we're lucky in the sense that with DLC you always want to put in a small amount of innovation each time to keep things fresh and avoid becoming a production line. So with the Kasumi DLC we've tried some new game elements, few introduce spy elements to the gameplay. That was fun for the internal team because it's a new set of challenges. As we do that sort of work between the two major projects it's a good way to test out new ideas.

At the same time we can never veer to far from understanding what made the game great. The hard thing internally is that everyone wants to make change but we have to recognise that a lot of things got us up to that 96 score on Metacritic. So it's very dangerous because any changes to the formula might be good and might be bad and you're walking this edge. We recognise that it's not a perfect game and there are still areas we have to work on. I think we're carrying forward the same kind of design philosophy that we did on the first one to the second one. When the first came out we broke down every bit of feedback we got, good or bad. And eventually we broke it down and addressed those issues. Really it's no different for the next iteration. It's about evaluating the work and the process and then making slight adjustments and testing it out as scientifically as possible. The project director and I went to the same engineering school so we think along the lines of a feedback loop. You have a hypothesis of the direction you'd like it to go, you put it out, see what the reception is, see how it actually ended up and then you make your subtle tweaks and come back and do your next iteration. Each time you're really leveraging against that foundation and time and investment you've put into it and hopefully edge towards higher and higher sales and critical response.

Adrien Cho is lead technical artist at BioWare. Interview by Matt Martin.

Related stories

Anthem delayed to 2019 - Report

BioWare devs tell Kotaku the studio is feeling the pressure of EA's expectations and YouTubers' anger

By Brendan Sinclair

Dragon Age creative director Mike Laidlaw leaves Bioware

The world of Dragon Age is "in good hands" as Laidlaw brings 14 years at Bioware to an end

By Matthew Handrahan

Latest comments (3)

Nik Love-Gittins Senior Character Artist, FreeStyleGames7 years ago
Interesting article. Nice to read about someone else's experiences in the same field as me.
0Sign inorRegisterto rate and reply
CaseyB Artist, Digital Extremes7 years ago
0Sign inorRegisterto rate and reply
Matthew Dai Business Development Associate, Adia Digital Art6 years ago
This is not a new article, but I'm glad I found it. Being on the service-provider side of outsourcing these are insights we are seeing more and more developers adopt. Good read.
0Sign inorRegisterto rate and reply

Sign in to contribute

Need an account? Register now.