Tech Focus: The Elder Scrolls V: Skyrim
Digital Foundry on the scale of Skyrim's technical accomplishment and the controversy over its many bugs.
The notion of consumers as beta testers appears to be growing increasingly commonplace, and it's something of a shame that Skyrim should be getting so much attention for its technical problems when the core package is simply remarkable. In this article, I'll be taking a look at some of Skyrim's most impressive technical achievements, but also highlighting related bugs and issues that are causing concern amongst the userbase. Could they by any chance be related? In some cases, perhaps so.
As the stack of YouTube bug videos rises seemingly by the hour, the question is, how can complex systems and features be fully tested when the circumstances in which they come into play will almost certainly change between players, and when the behaviour of the gamers themselves is so difficult to anticipate? Just how thorough can conventional QA be when we have a game as vast, open and as wide-ranging as this?
Make no mistake, despite the game's miniscule size in terms of compressed game data on the shipping DVD, there's no doubting the scale of the game and the sheer scope for adventure it contains. Bethesda's remarkable compression scheme ensures that the entire Xbox 360 version of the game weighs in at just 3.8GB. Never mind that we're now in the era of the 50GB dual layer Blu-ray: the Xbox 360 version of Skyrim in its entirety would fit snugly onto a single layer DVD with space to spare.
Skyrim's bugs are grabbing the headlines but just how thorough can conventional QA be when we have a game as vast, open and as wide-ranging as this?
With careful planning on how the files are laid out on the disc, Bethesda can ensure that the all-important texture and geometry data is always available on an area of the platter that offers the most throughput on the Microsoft console's 12x DVD drive (typically the outer edge of the disc), meaning fast streaming and minimal pop-in. The Xbox game hasn't been pared down that much either when compared to the PC version, which weighed in at around 5.5GB of data when we downloaded it from Steam. We see some occasional reduction in texture quality but everything else looks remarkably like-for-like.
For all the bugs and issues being reported about Skyrim, its fundamental achievements are simply staggering - starting with the environment itself. Bearing in mind the sheer size of the game and its myriad quests, our first thought is that the realm of Skyrim is actually procedurally generated - the result of a mathematical equation in the same way that the vast, expansive world of Codemaster's FUEL is generated. The notion is nothing new: 8-bit classic Elite generated eight entire galaxies in much the same way, and it could have churned out an infinite number of them if Braben and Bell were that way inclined.
In a recent podcast with Major Nelson (aka Microsoft's Larry Hyrb), Bethesda played down the procedural generation idea and simply mentioned that its compression was phenomenally good. Encoding design data into the height maps of terrain has been seen as one possible solution for storing enormous amounts of environmental data in a highly space efficient manner, but the developers are keeping their cards close to their chests on how it's actually done.
Skyrim manages to cram a huge world with visuals of this quality into a mere 5.5GB download on PC. Remarkably, the Xbox 360 version is a mere 3.8GB in total.
The creation of the landscape itself is just one of the technical marvels in the game - another concerns the amount of characters, objects and containers: thousands upon thousands of them spread across the sprawling map. This is something that certainly could not be procedurally generated without nerfing game balance, and may explain why save games incrementally grow as you progress through the game: it would make sense for the save to act as an ever-expanding "patch" that tracks how the objects and characters have moved from the default positions. The remarkable thing is that no matter how far you progress in the game you can return to the scenes right from the game's opening - and nothing has changed, even if days and weeks of in-game time have passed.
This may account for the reports we are seeing emerging of problems relating to the PlayStation 3 version of the game, where ballooning save-game sizes are being blamed for causing issues for some Skyrim players on the Sony platform.
Quite why it's just the PS3 game that is apparently affected is unclear, but it is the platform with the lowest amount of free RAM and it's clear from other issues that despite improvements from the Fallout 3 era, Bethesda are still having trouble making their PS3 work equivalent to the "lead" Xbox 360 version.
We were unable to replicate the save game issue thus far in our Digital Foundry analysis, but GI.biz's Dan Pearson - 18 hours into the PS3 version - seems to have experienced the problem and describes the increased performance after moving across to the 360 game as "revelatory".
Tracking a bug like this sounds like a nightmare, but according to Bethesda fans, the exact same issue exists in both Fallout 3 and New Vegas (some even saying that Morrowind on the original Xbox did the same thing) and questions are being asked on why the new game with its enhanced engine is exhibiting a similar issue.
So is Skyrim a victim of simply being too rich and too complex for its own good, was QA simply a little lacklustre, or was the game simply rushed to market?
Perhaps befitting the SKU that Bethesda has designed as lead platform, the Xbox 360 game doesn't seem to have quite as many problems as its PC and PS3 siblings, but the issue it has manifested is a bit of a doozy, with the preferred HDD install actually robbing the game of its ability to render the highest resolution textures. Bethesda initially denied there was a problem - perhaps understandable bearing in mind that HDD installation is an OS-level function that game developers have very little control over. Regardless, a later post on its official forum suggests that Bethesda are at least aware of this issue and plan to resolve it in a future patch.
Other bugs are manifesting in interesting and unexpected ways, and don't sound so much like QA oversights - more like unintended consequences of the open nature of the immense world the developers have created. Bethesda is making some bold claims for the intelligence of the NPCs in Skyrim, and the way that they interface with the game world. The "How to Steal in Skyrim" video soon went viral, currently weighing in at 1.5m views, demonstrating how you can thieve any object you want from anywhere by placing a pot over the heads of characters in the room.
It looks ridiculous, but it demonstrates that the NPCs are "seeing" the world and reacting to visual input - it's just that they're not smart enough to cope with out-of-the-box thinking from the player, and it would require some remarkably bright QA personnel to anticipate this problem. On top of that, while the AI may well be better than the average NPC's in many respects, it's clear they have trouble adapting to changing situations, resulting in abrupt changes of mind that don't make a lot of sense to the player.
The PS3 version of Skyrim - considered to be the weakest version - up against the PC edition of the game running at 720p.
Other issues seem to relate to the physics code being unable to cope with certain mixtures of objects, terrains and magical effects, resulting in some unintentionally hilarious consequences. Again, it's really difficult to imagine how even the best QA could anticipate some of the challenges conjured up by inventive players. Some might say that it's all part of the charm of a game that is so flexible and wide-open for experimentation.
So is Skyrim a victim of simply being too rich and too complex for its own good, was QA simply a little lacklustre, or was the game simply rushed to market? It's hard to think that it's not a combination of all three. For all the issues that are directly related to the game's cutting edge technology, there exists many more, particularly on the PC version where - depending on who you talk to - quests don't activate, mountains flicker in and out of existence, NPCs talk to you through walls, AI pathfinding can go completely bananas and horses can ride dragons. And that's all after a day one patch released for all three platforms that would have presumably quashed issues discovered after the game went gold.
Where does Bethesda go from here? Another patch is incoming, which the firm promises will fix as many bugs and performance issues as they can find. Will it stop the tide of unintentionally hilarious YouTube videos? Probably not. I certainly hope not. One of Skyrim's key strengths is that almost anything can happen and so long as nothing game-breaking happens as a consequence, we wouldn't want that to change...
I blame Havok more than Bethesda. You do not need a Bethesda game to make such videos, you need a Havok game; any will do.
Weightless corpses flipflopping around players' legs. The gravitational force being a linear constant with falling speeds not accelerating ever. Downhill rolling being approximated by horizontal acceleration. Surfaces having zero friction. The weightless bodies having elastic collisions with the ground wherein they retain all their energy and just change direction.
Imo, one cannot QA Havok physics, one can only argue against using them in the first place.
Posted:A year ago