Tech Focus: The Rise of FXAA
Digital Foundry on NVIDIA's ultra-fast, high quality anti-aliasing technology
Previously on Digital Foundry we discussed how morphological anti-aliasing (MLAA) has evolved from a theory put together by Intel into a working technology for PlayStation 3, developed by Sony's Advanced Technology Group (ATG) and is now evolving into a viable solution for both Xbox 360 and PlayStation 3.
However, MLAA isn't the only solution being worked on by the industry - far from it. Over at NVIDIA, technology developer Timothy Lottes has been working on an alternative to MLAA which is winning widespread acclaim. Lottes' FXAA technique is remarkable in that in its console iteration, it is able to process a 720p framebuffer using around 1ms of GPU time. Bearing in mind that the average console 30FPS game allocates 33.33ms to each frame, this means that anti-aliasing can be implemented without a radical impact on game performance.
Just like MLAA, FXAA is a post-process, screen-space anti-aliasing filter. This means that each frame is generated as per normal, with the AA code considering the image as one flat object for analysis. MLAA seeks out edges within the image and blends them. The advantage is that the edge-smoothing itself is nothing short of exemplary, but "false-positive" readings on what are edges and what are not can result in some ugly artifacting, especially in motion. Sub-pixel detailing can also look poor, with pixel-popping artifacts exaggerated rather than diminished.
In fairness, the Jimenez MLAA technique we looked at proved to be quite adept at skipping sub-pixel issues, providing a very clean look, but the approach that FXAA takes is an altogether different technique, and has been specifically developed to make the most of console and PC graphics technology.
Modern PC graphics cards are several generations ahead of the RSX and Xenos graphics cores within the consoles, meaning that FXAA comes in two distinct flavours - an ultra-fast rendition for the consoles (FXAA works on both Xbox 360 and PlayStation 3, though in some cases the 360 has a performance advantage) while the PC version, as author Timothy Lottes puts it, "is designed to take advantage of the feature and performance advantage PC GPUs have to provide very high quality."
But what is FXAA? How does it work and how is it different to the other post-process anti-aliasing technologies out there? Criterion's Alex Fry described it to us as a "heuristic blur filter", which chimes nicely with the author's own more detailed description.
"FXAA Console conditionally applies a blur filter which adapts in length and in sample locations based on a pixel's contrast and local gradient - effectively an artistically modelled intelligent blur filter," NVIDIA's Timothy Lottes says.
"FXAA Quality [for PC] re-samples the image by shifting each pixel's centre. Pixel shift is computed by a combination of an MLAA-like approach (which is limited to searching in either the vertical or horizontal direction) and an algorithm which detects possible pixel-sized or sub-pixel sized features… Visually the PC version is sharper and has higher quality on near horizontal and near vertical edges."
The new technique is intriguing in that Lottes targets FXAA to do more than just smooth edges and eliminate "jaggies". It targets anti-aliasing in a much broader sense in that it attempts to eliminate distortion effects that appear when a higher resolution image is rendered in a lower resolution. As Lottes says on his blog:
"Single-pixel sized features alias by snapping from pixel to pixel, and sub-pixel sized features alias by flickering on and off as they overlap a pixel's centre. To me, this is more visually distracting than edge crawling on multi-pixel features."
As you can see in these images, FXAA does a great job in isolating the high contrast pixels and blending them away. It's an approach to anti-aliasing that we've yet to see explored this completely in any other post-process technology we've looked at thus far:
The advantages of post-process AA are numerous. Deferred rendering techniques - which are becoming increasingly commonplace - do not play nicely with traditional multi-sample AA under DirectX 9 and while this was resolved to an extent in DX10 and DX11, the memory implications required can be quite scary and often unviable (put this way: there's a very good reason Dead Space and its sequel have no anti-aliasing - even on PC). From a console perspective, the bandwidth and RAM needs of MSAA are becoming too burdensome for developers looking to push their engines to the limits of the hardware's performance.
FXAA has already been implemented in various older forms (it icontinually evolving and improving) and has already been seen in PC titles such as Duke Nukem Forever, Crysis 2, Age of Conan and has picked up a number of glowing reviews from blogging game developers, hugely impressed with the results plus the quality of code itself, the tweakable variables on offer plus the extensive annotations accompanying the source.
It's clear that FXAA has a big part to play in improving image quality in a very efficient manner, and while all post-process anti-aliasing techniques will be challenged by sub-pixel detail, Timothy Lottes has been working on an interesting solution - to combine his technique with super-sampling, the process of downscaling the image from a higher resolution. Bearing in mind the sheer amount of untapped resources in many modern PC GPUs, it's a compelling approach.
"Applying FXAA before downscaling and averaging samples increases the quality of super-sampling, and with FXAA, fractional super-sampling works quite well: even when rendering only 1.7 times the number of pixels," he says.
"This enables the common deferred shading based game engine to transparently scale pixel quality along with GPU performance without going through the complexity of adding the variable rate shading required with hardware multi-sampling to maintain performance. An exciting real world application would be for a game to limit itself to 60Hz on the PC but use the left over GPU processing power to increase pixel quality using dynamic fractional super-sampling with FXAA."
Another exciting element of FXAA is the fact that it has been developed in a very open and collaborative manner. Timothy Lottes has posted a battery of source code revisions to his blog and has accommodated feature requests, bug-fixes and improvements suggested by game developers.
"Publishing preview source on my blog was a great way to get in touch with developers and get early feedback: feedback which has been instrumental in improving FXAA," Lottes tells us.
"Also from what I have seen, seems like developers are very open with their post process AA techniques. For example, the 2011 SIGGRAPH AA Course has a good representation from throughout the industry."
FXAA is also attracting positive comments from game developers in that NVIDIA has effectively developed this as a gift to the games industry, with Timothy Lottes confirming on Twitter that FXAA has a public domain license, meaning that anyone can utilise the code in their games.
Looking to the future, Timothy Lottes has his own views on where anti-aliasing is heading. More horsepower in the next generation of consoles should allow for more advanced techniques (perhaps along the lines of FXAA Quality) to be utilised.
"It's probably safe to assume that AA techniques which are only practical today on higher end PCs will be possible on next generation consoles," he explains.
"Given that AA quality is highly tied to the amount of GPU performance available per pixel rendered: assuming that next generation console GPU performance greatly increases while still targeting HDTV resolution, developers should have a lot more flexibility with regards to AA techniques. For example, I would be highly tempted to trade 1080p resolution for really good AA at 720p, however with 3D stereo I would go with the highest horizontal resolution possible, 1920, to maintain highest stereo separation detail, and possibly drop the vertical resolution. "