Tech Focus: Data Streaming and the Future of the Optical Drive
This week's release of Battlefield 3 presents a certain proportion of the Xbox 360 audience with an unenviable choice: buy a hard drive or other form of storage media device, or face up to playing a woefully inadequate version of the game. The arrival of BF3, along with other hard drive optimised titles such as id software's Rage suggests that the requirements of game development now outstrip the capabilities of optical drives.
Make no mistake, the difference between Battlefield 3 running with the "HD content" installed and running the game without it is absolutely remarkable. The game is awash with low resolution textures and it's difficult to imagine that DICE would want any one to play through the game with assets as rough as these.
"There's nothing magic about it," BF3 executive producer Patrick Bach told Gamerzines.
"It's the same thing we do for PC and PS3, so there's nothing extra. I think the controversy about this is that we actually let you do it on 360 for once. So what it does is it gives you the same abilities, kind of, as the PC and PS3. You can actually stream information from the hard drive. That's new for Xbox 360, but it's not a new idea for the gaming industry as a whole. No one has really tried to do it properly, so us doing it will create question marks."
Rage and Battlefield 3 get best performance from a 360 hard drive install - is the age of the optical disc nearing its end?
So why is the hard drive install necessary? The problem developers are facing is that the 512MB of RAM in current generation consoles simply isn't enough, so the solution is to stream in gameplay assets - visual and audio - on the fly. Up until recently, on Xbox 360, this has been achieved exclusively by using the optical drive.
The 360's 12x DVD drive can offer some very fast seek and transfer times, and development tools offered by Microsoft allow developers to strategically place content on different areas of the disc. The closer the laser moves to the centre of the disc, the slower data transfer becomes, so developers can prioritise content by moving it to the faster areas of the disc. Up until now, that has proved to be enough: even up against PlayStation 3 titles that partially install to the hard drive, the Xbox 360 has proven to be competitive in terms of streaming and loading times in general without requiring any mandatory installations.
However, as the demands on streaming have increased, the ability to keep high detail artwork streaming in quickly enough has now seemingly moved beyond the spec of the optical drive, necessitating the install of the most challenging material to a faster medium. With id software's Rage, there is also a very significant difference between the gameplay experience when played with and without a hard disk install: texture pop-in is a bit of an issue even in an optimal configuration, but it's so much worse when the HDD is not available.
Rage on PS3 also demonstrates quite convincingly that even when the hard drive is available, there are still some issues with performance. Responding to the gripes of PS3 Rage players, John Carmack recently tweeted that the level of RAM occupied by the PS3 OS in combination with the lack of unbuffered IO gives the Xbox 360 version of the game a significant advantage, something borne out in the Digital Foundry Rage Face-Off I completed earlier this month.
So, how does DICE's decision to go for optionally installed content work out for the 360 audience in particular? In short, there is going to be a section of gamers that only gets to see the game with visibly diminished visuals. Quite what the percentage is who don't own any kind of mass storage device remains unknown.
"The thing with the 360 is that you need to be able to give consumers a game where you don't have to install it on a hard drive, because there are 360s without a hard drive. So we need to give you the option of installing it, rather than just demanding it. You could call it a 'standard-def' version for the 360 if you don't have a hard-drive," explains Patrick Bach.
In fairness, Microsoft has offered up a lot of different options to ensure that BF3 users can easily benefit from the enhanced "HD" texture pack without having to shell out the cash for one of Microsoft's premium-priced hard drives. The additional content itself weighs in at around 1.6GB, so that means that even users with entry-level Xbox 360S "Slim" consoles should be able to accommodate BF3's lush visuals by installing directly to the 4GB of internal flash storage, with plenty of room left over for player profiles and LIVE downloads. [Updated: For reasons unknown, it appears that DICE has disabled installs to any device other than the hard drive]
Over and above that, Microsoft has also expanded the Xbox 360 platform by allowing for USB flash drives to be utilised as hard drive replacements.
As you can see from the table below, culled from Digital Foundry's extensive flash drive benchmarking on the Microsoft console, the raw throughput of a relatively inexpensive USB stick can match the 360's hard drive. Indeed, as there are no mechanical moving parts, seek times between files could actually be faster than a hard drive.
Only when we went a bit mad and used a cheap no-brand MicroSD card in a USB adaptor did we see a radically reduced level of performance, but in most of our benchmarks, the cheap 16GB flash drive we bought from Amazon was a sterling performer.
Halo 3: Level Loading Tests
|Section Tested||DVD||360 HDD||8GB MicroSD Card||16GB ByteStor||40GB USB HDD||128GB SSD|
So what does the Battlefield 3 and Rage experience tell us about the future of the optical drive? After all, when the next-gen consoles arrive, core game assets will be even more data-intensive. Well, we can safely assume that the drive will actually still be there: pressing optical discs - even dual layer 50GB Blu-rays - is going to be a lot cheaper than shipping game-specific SSD-style cartridges. It's also safe to say that broadband infrastructure will not be sufficient to support workable downloads in all current territories, though if all games don't ship digitally day-and-date with their retail counterparts, that would be hugely unfortunate.
The problem I have with a migration across to PC-style game installing is that it impacts the console's defining "plug and play" characteristic. Already we have plenty of complaints about this - especially on PlayStation 3, where playing a new game can be friction-filled experience consisting of system updates, patches and of course mandatory installs. In an ideal world, I'd like to see any kind of forced install banned from next-gen consoles, but bearing in mind the Rage and BF3 experiences, is this possible?
Well, it's safe to say that optical drive performance can radically improve. Let's assume that next-gen consoles will ship with Blu-ray drives: 12x speed BD-ROM units are now in the mainstream, offering an enormous boost to throughput when stacked up against the 2x drive that is standard in the PlayStation 3. There'll still be issues with seek times of course - but we can assume that these will be faster, and we'd hope that all systems would ship with a decent amount of flash memory, a mechanical hard drive - or, as is the case with the higher-end Xbox 360 SKUs in the here and now, both. If Microsoft can ship an entry-level 360 unit with 4GB of flash RAM, who knows how much cheaper storage will be when the next-gen console hits in 2013/2014?
Assuming that all consoles ship with some form of fast storage as standard, perhaps Naughty Dog's Uncharted 3 can offer up a glimpse of the future. The game effectively maxes out a full 50GB dual layer Blu-ray disc, cramming it with 46.3GB of data driven by a mere 18MB executable file.
Diving deeper into the make-up of the new game reveals some interesting facts. 22GB is taken up by movie files - with every cinematic rendered twice: once in 2D, once in 3D. That's a colossal amount of data, but it's still less than half of the space used on the disc, meaning a phenomenal amount of assets related directly to gameplay. In addition to that, we see a large number of files compressed with Sony's proprietary archive format too - so not only did Naughty Dog require a 50GB Blu-ray, they even needed to compress their data still further to make it all fit.
That is rather impressive, but the fact is that Uncharted 3 has no mandatory install whatsoever. The seamless experience it provides is achieved entirely through some extreme hardcore streaming, just like Rage and Battlefield 3, the different being that Naughty Dog had a single, fixed platform to develop on (though even that caused some issues, as this hair-raising and very revealing post-mortem feature by 1UP explains).
While every game design will have its own hardware challenges, Uncharted 3 demonstrates quite effectively that extreme streaming of game assets needn't necessarily mean mandatory installs. On-the-fly streaming can achieve a hell of a lot, and the benefits of a high-speed storage device in addition to the optical drive can preserve the "plug and play" console experience. As we move towards the release of the next-gen consoles, the make-up of the optical drive and the flash medium will come into focus. Hopefully a standard platform along these lines at launch will mean that sub-optimal experiences with certain hardware configurations a la Rage/BF3 will no longer occur...