Under the Bonnet: SHIFT 2 Unleashed - 2
Slightly Mad talk AI, Helmet Cam and capturing the thrill of the race
In this concluding segment of Digital Foundry's extensive chat with Slightly Mad Studios on the development of SHIFT 2 Unleashed, we discuss topics as diverse as artificial intelligence and steering wheel peripherals, as well as more technical issues such as the move to a light pre-pass deferred lighting system and the limitations of the current generation consoles. Also coming into focus is the studio's attempt at capturing the visceral thrill of real-life racing via innovations such as the unique Helmet Cam.
Once again, many thanks to the following team members at Slightly Mad for the truly remarkable level of effort they put into making this feature happen: Andy Tudor (Design Lead), Ged Keaveney (Lead Programmer), Tim Mann (PS3 Lead), Jan Frishkorn (Vehicle Lead), Andreas Moll (Art Director), Doug Arnao (Physics Lead), Dr Stephen Baysted (Audio Lead), Rob Dibley (Render Lead), Peter Nicholls (Render), Stephen Viljoen (COO), Suzy Wallace (Producer).
We're returning to the interview right where we left off, so for those new to the party, part one of the feature can be found here.
Q: Some driving games receive a lot of criticism for the quality of the AI. What are the challenges in creating believable CPU opponents and how did you implement them in SHIFT 2?
Andy Tudor: The 'X' of Need for Speed: SHIFT really focused on 'the driver'. So when we were implementing the XP system we analysed all the actions a driver does on the track - so overtaking, drafting, taking the inside line, getting a good start off the grid etc. From those it was then pretty clear they fell into two categories - aggressive and precise. When we then looked at real-world drivers this 'aggression versus precision' was substantiated in their personalities - Schumacher vs Villeneuve or Gronholm vs Blomqvist.
When it came to the AI therefore, their profiles were created using traits that not only defined their raw 'ability' but their personalities too. So things like 'Risk Taking', 'Concentration', 'Grudges', 'Antagonism' and 'Adrenalin' were all present. We then made around 15 unique profiles that dialled up certain attributes - so there'd be an AI who was really clinical and avoided all crashes but then there'd be another who'd actively bash into you, one that would swerve out of your way if you drafted him, and others that made silly mistakes when cornering etc.
Within each race therefore you'd have a variety of drivers with different levels of ability and different reactions to you as a player. These were then playtested and tweaked and scrapped and redone and boosted right up to the last minute; the anti-vision being a funeral procession of cars that trawled around the racing line but one that recreated the kind of actions you'd expect to see playing others online - true gritty back-and-forth duelling.
When we were implementing the XP system we analysed all the actions a driver does on the track... overtaking, drafting, taking the inside line, getting a good start off the grid etc.
As we looked to community feedback after the release of Need for Speed: SHIFT however, we were given another perspective on the AI: players were having a great time racing only to be knocked off the track on the last lap by an aggressive driver and lose. Or they'd be driving side-by-side with the AI and they'd be trading paint with them in an effort to knock them off the racing line. Or they'd be approaching a corner and the car in front would spin out for no apparent reason.
In many cases, either an overly-aggressive opponent was ruining your race or they were appearing to do 'silly' things. Because there were a wide range of personalities in each race however these incidents were limited to a small percentage of the AI pack and instead of players identifying some as aggressive and some as precise, instead most were 'correct' and the rest were 'bugs' or 'idiots'.
So going forward into SHIFT 2 Unleashed, our aim was to remove the 'idiots' with a three-pronged attack... To streamline the number of traits that created their personalities in order to reduce the randomness effect, dial down or even remove the traits that caused either bizarre or harmful on-track manoeuvres, and focus more on their 'abilities' (defence, blocking, intelligence, passing, reactions) to achieve consistency and calibre within our AI. When you couple this with some matchmaking filtering to ensure 'aggressive' human opponents don't ruin the online experience too we think the pack racing experience in SHIFT 2 Unleashed is a lot of fun and so far, the reaction has been overwhelmingly positive to the changes.
Q: Helmet Cam is a key addition to SHIFT 2; what did you set out to achieve with this and what problems did you have to overcome?
Andy Tudor: When we approached the cockpit cam in Need for Speed: SHIFT we knew it had to be fixed. Here's the most natural view for a racing game and no-one uses it. Why? Well a number of reasons - not all games have it for a start so you get used to going from one game to another using another (the favourite being Chase Cam). Also, it doesn't have a clear advantage like the others; the bumper cam gives you an enhanced sense of speed because you're very low to the ground, the chase cam gives you a greater sense of your surroundings, and the hood cam is a compromise giving you a great sense of speed plus the spatial awareness of how wide your car is. The cockpit cam therefore was kind of relegated to fourth place and it was very restrictive - with the area above the windscreen and the dashboard below it, it felt like driving with immovable widescreen borders and a smaller view of the world outside as if looking through a telescope.
We used a variety of techniques therefore to overcome these issues. The first was to allow you to look around the cockpit. We were crafting each one individually and the car artists were putting a ton of detail into them. With the lighting effects and variety of materials in use and the animations of the driver, this was a no-brainer to showcase this attention to detail. It also gave you freedom from being locked into that blinkered widescreen view - you didn't feel as constrained any more.
The next was to couple the camera to the physics in order to simulate the g-forces your body undergoes when accelerating and braking. This was key to the X of the "True Driver's Experience" and once again gave you a sense of freedom within those widescreen borders since they would now grow and shrink as you moved forward and backward in your seat.
We then used a variety of stylistic techniques from the first-person genre to further enhance the cockpit experience: at high speed the cockpit would blur out (similar to looking through the iron sights of a gun in Call of Duty) which really made your eyes focus on the road rather than the surrounding interior, and when crashing we simulated the disorientation of the crash by de-saturating the screen and blurring the edges (again, similar to a grenade going off).
The overall effect therefore was one where the player's perspective was constantly changing and was representative of both the physical and emotional experience you have when sat in the driver's seat. As with most new features of course, the strength of each component was raised and lowered and focus-tested throughout development ranging from the game being unplayable and people becoming motion sick to it being so subtle it was barely noticeable.
Q: So Helmet Cam is essentially an evolution of this work?
Andy Tudor: Aware that competitors were likely to copy it, we were eager to keep innovating in this space and so for SHIFT 2 Unleashed we wanted to take it to the next level with the Helmet Cam. By attaching the camera to the driver's head rather than their body we were able to achieve an even more authentic and immersive perspective on the action - one that now leaned laterally when approaching corners and tilted to counter-balance camber. We also used depth of field effects to draw attention to the apex of the corner so it became a great learning tool too. If Cockpit Cam was akin to a first-person shooter then this was pulling inspiration from flight sims.
Initially we included a reflective visor with the Helmet Cam too - but we found that the additional reflections caused too much of a distraction when racing. The perception was that in reality your eyes tend to look 'through' the visor and don't even register it when racing. In the game however it ultimately felt like driving through dirty glass though so it was removed.
Since the camera was now adjusting when approaching corners we also had to do careful balancing so that it felt natural and was complimenting where your natural eyeline was going. Initially it was quite distracting - the equivalent of driving a tank and rotating the turret whilst attempting to keep going in a straight line. A set of simple parameters allowed us to tweak the settings rapidly and allow the team to iteratively re-test and we nailed the behaviour fairly early on in the project.
In the same way that MP3 compression removes subtle detail from audio to compress it, we analyse the entire frame context and if there is a lot of foreground detail you won't miss background subtleties.
Q: We've talked in brief about the deferred rendering in SHIFT 2. Can you go into more depth on this? Bearing in mind the 4x MSAA we see on Xbox 360, is it safe to assume you're working with the light pre-pass approach?
Tom Nettleship: We use a three phase light pre-pass approach for our deferred renderer. The main reason we chose this as opposed to a two-phase approach was because being a racing game, we needed to focus on high quality rendering of diverse material types, ranging from rubber and cloth all the way up to paintwork, glass and carbon fibre. While a BRDF-lookup texture approach allows for decent material variety in a two-pass deferred renderer, it doesn't cut it when you need to implement a totally different lighting model (carbon fibre, brushed metals).
Also, our most important visual components are the cars, which need high quality environment mapping. We tried every encoding possible for the normals channel of the G-buffer before deciding that it wasn't possible to get an acceptable quality level for bodywork reflections. So, we dropped back to the cheapest normal encoding (888 view space XYZ, which helped a lot with PS3 performance) and use those for general lighting. For bodywork reflections we re-evaluate the normal mapping in the third phase of the light prepass render. It's more expensive, but the quality you get makes it worthwhile.
Q: With the increased flexibility in how you can use light, how does that benefit the player? A lot of gamers hear about techniques like deferred rendering but aren't sure about how it makes for a better game.
Andy Tudor: With a new lighting engine it's meant we've been able to get both 'temperature' and additional gameplay into the game. Lighting is one thing - talented artists can easily recreate a stunning sunset or a starry sky, but with the advanced techniques our engine is capable of we've now been able to temporarily blind the player with dazzling sunlight as they approach a corner or cast individual shadows across their dashboard when racing at night.
Not only do these effects look spectacular but they also invoke an emotional reaction from the player; concentration when approaching a tough corner, fear when descending into the pitch black darkness, or apprehension seeing the headlights of a car behind sweeping across your dashboard knowing that one wrong mistake and they're going to overtake. So lighting in SHIFT 2 Unleashed is an integral part of heightening the challenge when racing, not just making the environments 'look pretty'.
With 100 different track locations in the game spanning 15 different countries we also wanted to make sure they were both distinguishable and true to their geography for easy recall by the player. Regardless of track layout the sun and 'temperature' of the West Coast Willow Springs track therefore intentionally looks distinctive to that of the East Coast Laguna Seca despite sharing common geological traits such as ground composition and vegetation. As you play through Career therefore, a forest-based Belgian track immediately followed by a forest-based Italian track will look different to each other and ensure individuality and variety.
Q: You opted for MLAA on PlayStation 3 - is this the standard code supplied by SCEE's ATG as part of the PlayStation Edge tools? How easy was it to integrate into your engine?
Tim Mann: It's a slightly modified version of the pre-release Edge code. It's very easy to integrate in the pipeline and is placed between the HDR tone-mapping phase and motion blur. It can be quite fussy tuning-wise but we managed to find a good balance between too much edge detection - which produces too much blurring - and not enough, which leaves you with jagged edges.
Q: Overall consistency in terms of performance is very good in SHIFT 2, even with plenty of cars on-screen. What is your approach to ensuring a steady frame-rate even factoring in so many different rendering challenges?
Tim Mann: We take the view that both graphical and gameplay quality are critical to the experience of our games so we throw everything possible in, and then optimise via a range of in-house tools.
I don't think there was ever a silver bullet; rather every area was optimised individually. We used a number of approaches including controlling level of detail on light shaders, clipping light cones to geometry, running large chunks of the rendering logic on SPU on PS3, dynamically prioritising shadows based on light occlusion tests, MLAA instead of MSAA on PS3 etc.
Asset-wise, we profile the render cost of each vehicle (and each upgrade thereof) in a fixed scene in order to determine which vehicles need to be modified for performance and similarly we profile each track location using an automated camera system that moves around the track monitoring performance and identifying hotspots for optimisation.
We also use dynamic variation of detail in order to keep the performance up when it starts to drop due to a heavy load from some particular situation. In the same way that MP3 compression removes subtle detail from audio to compress it, we analyse the entire frame context and if there is a lot of foreground detail you won't miss background subtleties. In the same way that if you are driving very slowly you notice a lot more detail.
Q: There's been a fair amount of comment from the fanbase on what's been described as an inconsistent feel in the analogue controls that wasn't there in the original SHIFT. Are you aware of what is being discussed here, and do you think it should be fixed?
Stephen Viljoen: Immediately after launch we played with the community and heard their feedback on this issue, yes. After investigating it further - on a combination of platforms and input devices - we managed to isolate the cause and issued a fix for it in the console update/PC patch.
Q: There's some pretty serious wheel support in SHIFT 2, with a far greater list of devices supported on PS3. How important do you consider a wheel to the SHIFT 2 experience, and do you think there's a gap in the market for a higher-end peripheral for Xbox 360 owners?
Andy Tudor: New input devices like Kinect or PlayStation Move aim to break down the barrier between the player and the game by making the interface to controlling them as natural as possible.
Racing games have had their most natural interface for years now though - the same input device that's used in real-life cars: the steering wheel. Although we recognise that a majority of our players will be playing via a gamepad (same as most StreetFighter players will not own a Hori stick), the steering wheel/pedals/gearstick input method is one of our top priorities and so we test extensively on the major products out there currently.
In terms of the 360 specifically, the official wheel is a good starting recommendation and can be picked up at a reasonable price but we'd love to see a more durable 'Pro' version with detachable gearstick and a clutch pedal!
The initial contact point for Need for Speed: SHIFT was Patrick Soderlund at DICE. He's a massive racing fan and a successful Team Need for Speed driver in GT3 and winner of the Dubai 24H race.
Q: There have been leaps and bounds in cooperation between the different parts of Electronic Arts - for example, DICE helping to create the Hot Pursuit game world. In what ways were other EA studios able to help with the development of SHIFT 2?
Andy Tudor: The initial contact point for Need for Speed: SHIFT was Patrick Soderlund at DICE. He's a massive racing fan and a successful Team Need for Speed driver in GT3 and winner of the Dubai 24H race.
He was a huge supporter of our approach on that title; to make the first game that really captures the emotions of racing a car at high speed. During that transitionary first title we collaborated with the original BlackBox team that had worked on the previous NFS titles and they aided us in a number of areas including user interface and online support.
Going forward with the sequel we took control of those areas internally but then made new contacts at EA Vancouver in order to push our own ideas out across the franchise such as the Need for Speed VIP program and ensure consistency of the branding across the various NFS titles (Hot Pursuit, The Run etc).
As a key part of the initial SHIFT 2 Unleashed brainstorming we wanted to include a suite of features that allowed players to talk to each other, compete, compare times and see what the community was doing on a larger scale. We called that 'The Driver Network'. We were then put in contact with Criterion who, ironically, were developing a similar system called 'Autolog' for their upcoming Hot Pursuit title. Realising that we both had the same idea we then collaborated in order to meld the two together - the initial version being available in Hot Pursuit and then SHIFT 2 Unleashed building upon those with our additional features ('access anywhere' navigation method, video sharing, Records tab, additional recommendations, rewards list, automated messaging etc).
Q: Turn Ten has talked about making their game better by adapting existing ideas - for example, including Codies' flashback instant replays, and I believe Criterion's image-based lighting is in Forza 4. To what extent would you subscribe to this way of going about things?
Andy Tudor: To a certain extent... I think you have to pick and choose what's right for your game and how your players are playing it. We considered the replay feature ourselves for example but chose not to include it since it's a 'Get Out Of Jail Free' card. If you mess up on a corner then you idly hit that button and retry. Sure, it can save your bacon occasionally if you made an accidental mistake but having a handy 'reset' button does nothing to enhance your apprehension of crashing; there's no consequence if you can just undo it as if it never happened.
We would rather let you fully experience the visceral crashes we have in the game - the full damage model, the g-forces and stylistic disorientation effects etc - and then provide a gameplay balance to still allow you to get back on the track and claw back that lost position. Both methods basically aim to avoid the player hitting restart but it didn't feel appropriate for our game focusing on the driver and their battle with opponents on the track. Its unavailability in online modes anyway sealed the deal for us.
Q: And you have the same approach to technological innovation?
Andy Tudor: When it comes to tech, we always look at it from the player's perspective... how is deferred rendering going to affect my gameplay? In the case of SHIFT 2 Unleashed it means you get a truly terrifying, realistic, and beautiful night racing experience. When it comes to backend servers and the messaging to and fro between players via Autolog it means you play the game socially even when your friends aren't online.
I think the key to developing a successful franchise is community and innovation. With each iteration you have to keep the great stuff and enhance it, cut the stuff that didn't work or no-one is playing, fix or balance the existing stuff that's been nagging you, and then add a whole bunch of new stuff that your players are crying out for or has never been seen before. Get it in early and playtest/polish it until release then support the community after release and include them in discussions moving forward.
If you're only ever adapting existing ideas then you'll constantly be playing catch-up and players will simply see version 2.0 of a feature they've seen before.
When it comes to tech, we always look at it from the player's perspective... how is deferred rendering going to affect my gameplay? In the case of SHIFT 2 Unleashed it means you get a truly terrifying, realistic, and beautiful night racing experience.
Q: Any thoughts on some of the features we've seen elsewhere - such as dynamic time of day, weather, and 3D?
Andy Tudor: Dynamic time of day and weather were both in our original wish list for SHIFT 2 Unleashed but were postponed early on since they snowball into other features we would need dedicated time on to deliver to a 10/10 standard; if the weather changes for example during a race then you'll need to change your tires. That means pit stops need to also be included.
If dynamic time of day is available then it allows us to have really long endurance races (which also require pit stops) where we can more accurately simulate real series that require the swapping of two drivers. That then hints at some sort of team management therefore where it'd be great to add qualifying too. So they've been discussed definitely but didn't specifically fit this particular title.
When it comes to 3D, the original Need for Speed: SHIFT supported 3D on PC with the aid of the GTX series of NVIDIA cards. Racing games are perfect for this extra dimension due to their nature of the player moving rapidly through the environment in the z-axis. When viewing from the cockpit perspective particularly you get that extra spatial awareness of the interior and your proximity to the wheel, dashboard and the world beyond.
Q: Is there any degree to which you feel you're being held back by the current generation of consoles? As SHIFT developers, what would you want to see from next gen hardware?
Ged Keaveney: Processing power on PS3 is split between the SPUs and a dual thread CPU. We have a cross-platform small jobs system that is specifically designed to send tasks to either extra cores/threads on 360/PC or the SPUs on PS3. The SPUs are exceptionally fast but not all sections of logic are suitable candidates to run on them which in turn puts extra load on the single PS3 CPU that can be problematic. On 360 and PC we can send these larger jobs to other cores and run them with little difficulty though.
Ideally from a developer perspective, we'd prefer a more consistent architecture across all platforms to help reduce development time and platform specific issues, such as the multiple core approach used by 360 and high end PCs. Keeping the SPUs is fine but having a better balance between them and traditional CPUs would be better.
On PS3 and 360 we would like to see more memory and for it to be faster memory too. 512MB of memory is not enough compared to the capabilities of current console hardware. Also a lot of time is spent making sure assets fit into this restriction, time that could be better spent on polishing the game, especially considering the relatively cheap cost of memory.
From a rendering point of view I think right now the main thing we'd like to see is a bit more performance headroom on the GPU side of things as we're approaching the limit of what we can push on the hardware of the current gen of consoles. This would allow us to consider more costly approaches to things like anti-aliasing and introduce more transparency on vegetation. Ultimately it'd be wonderful if the next generation of consoles allowed for efficient rendering through a fully programmable pipeline but it feels a little early to say what could be achieved with this on modern hardware (certainly things like bokeh, order independent transparency and better reflections come to mind).
The nice thing with the current gen is that after so much time by so many developers spent on them we're really getting a lot out of the hardware and with a similar amount of time on DX11 class hardware we'll achieve some amazing results.