Valve's Michael Abrash: Latency is getting in the way of VR

15 or even 7 ms could be upper limit for believable tech, says developer

Valve's Michael Abrash has poured cold water on the idea of an imminent future containing believable virtual and augmented reality systems, writing that current limitations on latency are the bottleneck to future development.

Writing on the Valve Blog, Abrash discussed the problems caused by the delay between user input and display response, which causes particular problems in virtual and augmented reality systems because of the nature of the interface.

"Assuming accurate, consistent tracking (and that's a big if, as I'll explain one of these days), the enemy of virtual registration is latency. If too much time elapses between the time your head starts to turn and the time the image is redrawn to account for the new pose, the virtual image will drift far enough so that it has clearly wobbled (in VR), or so that is obviously no longer aligned with the same real-world features (in AR).

"How much latency is too much? Less than you might think. For reference, games generally have latency from mouse movement to screen update of 50 ms or higher (sometimes much higher), although I've seen numbers as low as about 30 ms for graphically simple games running with tearing (that is, with vsync off). In contrast, I can tell you from personal experience that more than 20 ms is too much for VR and especially AR, but research indicates that 15 ms might be the threshold, or even 7 ms."

Abrash continues to examine the problem in more detail, before coming to the conclusion that current minimum latency is around 36ms: "a long way from 20 ms, and light-years away from 7 ms."

"Short of racing the beam, there is no way to get low enough display latency out of existing hardware that also has high enough resolution, low enough cost, appropriate image size, compact enough form factor and low enough weight, and suitable pixel quality for consumer-scale AR/VR.

"Someone has to step up and change the hardware rules to bring display latency down. It's eminently doable, and it will happen - the question is when, and by whom. It's my hope that if the VR market takes off in the wake of the Rift's launch, the day when display latency comes down will be near at hand."

The Occulus Rift, which completed a highly successful Kickstarter campaign last year, hopes to reinvigorate virtual reality when it launches in early 2013. For an insight into the device, read our interview with inventor Palmer Luckey here.

Related stories

Nintendo plots longer lifecycle for Switch

Firm wants to extend beyond the standard 'five or six years'

By Christopher Dring

Google prepping game streaming service - Report

Offering codenamed Yeti would work either with Chromecast or potential Google console

By Brendan Sinclair

Latest comments (6)

Peter Freese Senior AR Developer, Unity Technologies5 years ago
This seems solvable by building in some short-circuit tracking directly into the device front end. On the back end, the scene is rendered larger than the view field by some amount (say 50% larger). As the user turns his/her head, the display device calculates the difference between the current orientation and the rendered scene center orientation, and shifts the viewport accordingly. Normal display buffering/latency can continue; as long as the user's head rotation velocity doesn't exceed the ratio of additional-render-buffer size vs. back-end-latency.

One catch with this solution is that normal view frustums won't work - you'd need some kind of spherical projection or view-plane correction.

Edited 1 times. Last edit by Peter Freese on 2nd January 2013 5:02pm

4Sign inorRegisterto rate and reply
Andrew Jakobs Lead Programmer 5 years ago
What nonsense is he talking.. We've been using VR for a long time already and it never was a problem, I guess people are becomming more snobbisch these days.. I still enjoy playing with the VFX1 (which for me has an even better tracking as the VR920 which I also have and where the tracking became more awfull with every new update (looks like they fixed it to some point in the US)).. We all know there is excellent trackinghardware and software, we all know displays are fast enough to render it, I mean, I can move faster sometimes with my mouse than with my head and the 3D monitors have no problem in keeping up.. And if we have to believe the word on the Rift, it is A LOT better than the VFX1 or VR920, so I don't understand what the fuzz is about, except for some snobbisch person for who it's all too slow..
0Sign inorRegisterto rate and reply
but he is speaking as if steam and online is needed as if they are now the only way to game. VR could be done first via single players games, and LAN parties, etc The bottleneck of the internet and steam is simply the age old problem of tech and infrastructure, which has little to do with the realization of VR. Believe it or not we gamed very well before everything had to be online and or via steam.

So while I agree latency is an issue, its only a temporary and seperate one that will be fixed with better tech and infrastructure, meanwhile VR tech can and should continue to move ahead . Bringer us all closer to those virtual worlds we all crave.

Edited 3 times. Last edit by Todd Weidner on 2nd January 2013 7:05pm

0Sign inorRegisterto rate and reply
Show all comments (6)
Wow Andrew, pretty hardcore - I found the VFX1 ( near impossible to wear, and as it did not come with installed tracking, we assume you jury-rigged your own compromise?

The latency issue seems to be a factor of concern with all that had looked with a professional eye at the rough Rift prototype last year - though I am confident that the Oculus guys have a handle on the resolution and latency concerns voiced.

The tracking is another issue altogether; I know you may think just because Intersense and Polyphemus are available, but they are expensive solutions to bolt on the dev kit let alone license for the final development Oculus Rift proposal. Sweeping generalizations at this point could hamper the final delivery within the $300 price point promised. I would also avoid the claims that you can just "...hock it up to your Kinect.." that I read on one forum - not going to happen without serious agreement.

I wonder how many of those following the Oculus Rift campaign will be prepared for this to be a PC only system - especially as no Gen-7 or Gen-8 console could handle the requirements of the system unmodified?

Even with all the hurdles to clear, this is still the most exciting development in the sector! Good luck Oculus.

Edited 2 times. Last edit by kevin williams on 3rd January 2013 2:01pm

0Sign inorRegisterto rate and reply
I had a question last year about what alternative VR HMD's are out there. For some reason I never responded (must have been too many Christmas parties). Anyway:

- The CyberMind Visett is a UK designed VR HMD, built for professional and commercial entertainment use and shows the separation from the proposed dev kit and the reality of fielded VR system in the professional market - it retails for $12,690**

** - price dose not include tracking, nor software

Edited 1 times. Last edit by kevin williams on 3rd January 2013 2:46am

0Sign inorRegisterto rate and reply
Roberto Bruno Curious Person 5 years ago
@Todd Weidner: No, actually he isn't speaking about online gaming at all.
What he's talking about isn't "net latency". He's talking about input latency on locally-running software (aka single player experiences).

Edited 1 times. Last edit by Roberto Bruno on 3rd January 2013 9:14am

0Sign inorRegisterto rate and reply

Sign in to contribute

Need an account? Register now.