Tech Focus: Game Graphics vs. Movies
Does resolution really matter? Developers discuss bringing a filmic look to next-gen titles
When thoughts turn to next-gen console technology, we seek to quantify the leap forward with absolute metrics - and resolution inevitably gets prominence. Looking at the current generation of consoles, technical requirements for Xbox 360 games at launch (which were quickly overlooked) suggested that games should run at a minimum of native 720p with 2x multi-sampling anti-aliasing. A reasonable expectation for next-gen is full-on 1080p with the equivalent of 4x MSAA, but to what extent does resolution actually matter?
An interesting discussion kicked off on the blog of NVIDIA's Timothy Lottes recently, where the creator of FXAA (an anti-aliasing technique that intends to give games a more filmic look) compared in-game rendering at 1080p with the style of visuals we see from Blu-ray movies.
"The industry status quo is to push ultra-high display resolution, ultra-high texture resolution, and ultra sharpness," Lottes concluded.
Do 1080p games super-sample compared to Blu-ray movies? Is the current focus on high contrast, high detail artwork the right approach for a more filmic next-gen?
"In my opinion, a more interesting next-generation metric is can an engine on an ultra high-end PC rendering at 720p look as real as a DVD quality movie? Note, high-end PC at 720p can have upwards of a few 1000s of texture fetches and upwards of 100,000 flops per pixel per frame at 720p at 30Hz."
Comparing screengrabs of a game (Skyrim running with a super-sampled anti-aliasing hack) with the Robert Downey Jr Iron Man movie, the NVIDIA man reckons that even at native 1080p with no MSAA, game rendering is still effectively super-sampling compared to the quality we see in theatrical presentations, and maybe game developers could pursue a more filmic look using fewer pixels in concert with other processing techniques. Lottes noted that there is little or no single pixel-width detail in 1080p Blu-ray movies, as we can see in spades in ultra-precision PC presentation, suggesting that the same level of detail can be resolved in gaming without recourse to a 1080p framebuffer - or else utilising 1080p with a lot of filtering that gives the illusion of a lower resolution.
The notion was endorsed by many games developers, with DICE's rendering architect Johan Andersson saying that "It's not about the amount of pixels, it is about the quality of the pixels and how the overall (moving!) picture looks like. Less aliasing = less noise for your brain to interpret = more pleasing and easier to see visuals."
"Something else that bothers me in most games these days is how much contrast there is in the textures," added Prey 2 lead graphics programmer Brian Karis.
"Having physically based material guidelines help but the artists seem to try everything they can to create higher contrast. The result in my opinion is crunchy, noisy and often nasty looking images. I'd call the status quo ultra sharpness, ultra contrast."
A good example of the approach being described here can be seen in a great many Unreal Engine 3 titles - with the likes of the Gears of War games and Enslaved cramming a phenomenal amount of high frequency detail into a 720p framebuffer.
"We do what is essentially MSAA. Then we do a lens distortion that makes the image incredibly soft (amongst other blooms/blurs/etc). Softness/noise/grain is part of film and something we often embrace. Jaggies we avoid like the plague and thus we anti-alias the crap out of our images," added Pixar's Chris Horne, adding an interesting CG movie perspective to the discussion - computer generated animation is probably the closest equivalent gaming has in Hollywood.
"In the end it's still the same conclusion: games oversample vs film. I've always thought that film res was more than enough res. I don't know how you will get gamers to embrace a film aesthetic, but it shouldn't be impossible."
To achieve a resolution equivalent to today's full HD Blu-ray movies, it therefore follows that the processing requirements could fall quite dramatically from what was previously thought. You don't need MSAA, you don't need super-sampling. While the image may fall foul of pixel-counting in that it doesn't hit native 1080p, precious rendering resources can still be deployed elsewhere.
"A game could render at native 1080p resolution (with extra optional hardware AA or SSAA) then use a fast post effect filter (optionally combined with post AA to "estimate" shaded samples which don't physically exist) to apply a film-like depth of field and get a result similar to a 1080p BR film," Timothy Lottes added. "And that this process would have a filter kernel which would be at least two to three (of 1080p) pixels wide (and likely larger in practice to get a good window)."
The ambition of what is being suggested here shouldn't be understated. It would involve a lot of buy-in not just from the game-makers, but also from the audience themselves who have become conditioned to a particular aesthetic. The perception of video games graphics in the HD era is that there's a purity to the look, a certain pristine edge to the visuals that has only been challenged by a very small number of releases (Limbo and Killzone 2 are two examples that spring to mind).
Even in some existing console titles, there's already evidence that the focus on detail is not so important and that rendering resources can be better deployed elsewhere
To truly work it would also need to be accompanied by technologies that make the most of the "freed up" processing budget. Lighting technologies in particular would need to make a step-up, but other elements such as animation and direction would also need to match. To get the filmic aesthetic, it's as much about how a game moves as well as how it looks.
However, in terms of the quality vs quantity argument, there is already a range of evidence within the existing console generation that the focus on detail is not so important and that rendering resources can be better deployed elsewhere. Call of Duty is perhaps the most spectacular example, if not quite along the lines being suggested by Timothy Lottes. It's a fact that the biggest example of the triple-A philosophy does not render at a recognised high definition resolution: a typical 720p game offers an additional 50 per cent of resolution over the 1024x600 framebuffer utilised in Modern Warfare 3 with the developers using the resources available to operate at a target 60 frames per second.
Other examples do tie in more closely with the ethos being suggested by Lottes: that it's not the pixel count that's important to image quality rather than the way those pixels are used. Namco Bandai's Japanese teams have experimented a great deal with resolution on current generation systems, most notably in Tekken 6. Without motion blur active, the Xbox 360 version renders natively at 1365x768, downscaling to 720p - using super-sampling to give some anti-aliasing. However - perhaps surprisingly - with motion blur active, resolution drops to 1024x576, but the game measurably looks better. Not only does the blur add more realism to the movement, but more detail appears to be resolved in the textures even though base resolution is effectively halved.
Discussing Tekken 6 in particular, RedLynx's Sebastian Aaltonen - the main technical mind behind the brilliant Trials HD - discusses how the lower resolution mode offers up system resources for other forms of graphical processing:
"Both configurations fit well inside the 10MB eDRAM. The 1024x576 is kind of a strange choice, as it's only around half the pixels of the 1365x768 and the cost of the blur filter comes nowhere close to the performance gained from the resolution decrease, and they are not eDRAM limited either," Aaltonen observes.
"The resolution reduction itself is not something I consider strange, but a reduction this large means they have something else going on than just the motion blur. The better texture detail you are seeing could mean they have enabled anisotropic filtering for the lower resolution."
What Timothy Lottes is suggesting is something a whole lot more ambitious - a new approach to rendering from the ground upwards, encompassing both the engine and the creation of the core art assets.
A more dramatic current generation example would be Remedy's Alan Wake. The game actually operates with an even bigger resolution penalty than Tekken 6, with a native 960x544 framebuffer - but the combination of 4x MSAA, phenomenal lighting effects and post-processing work ensures that the game does not suffer for it: aliasing and the dreaded "jaggies" are not especially an issue (though screen-tear definitely is). Remedy itself says that Timothy Lottes' FXAA is utilised in the American Nightmare Xbox Live Arcade sequel, most likely in place of the 4x MSAA of the original - so it will be interesting to see if resolution is increased, or if the processing resources are instead used on cleaning up the intrusive lack of v-sync.
As game developers transition across to a new breed of architecture, exciting possibilities open up - the ability to present gameplay in ways we've never seen before. There's been a lot of discussion about how to make the next-gen matter - this is just one discussion, one possibility. Developers like Crytek have already indicated in their SIGGRAPH and GDC presentations that the ability to innovate with new rendering paradigms is limited on current generation platforms. New hardware means new opportunities, and the notion of key developers, artists and decision makers already discussing the possibilities freely and publicly is another credit to the openness and the collaborative spirit that flows through the video game industry.