Microsoft developers to reduce Kinect lag

Researchers use aggregated data to improve accuracy and speed of body recognition

Microsoft researchers have issued a paper detailing potential improvements in the accuracy and speed of Kinect's sensor routines, although it is unclear if and when the findings will be implemented.

A summary of the paper, and link to the full text, appears on website I Programmer and primarily involves new AI learning routines that can be used to improve the device's ability to discern and track body parts.

Written by Microsoft Research in Cambridge, the labs experiments involve millions of 3D depth maps already identified with body parts. By aggregating the results a 1000 core server was able to create a series of data trees, which could then be used to assign a predicted body part to each piece of 3D pixel data.

The use of the aggregated data allows Kinect to estimate joint positions in a player's 3D model much more quickly than is currently the case. As well as removing the need for a formal calibration the new technology could also be used to reduce lag times from the currently relatively high levels.

The paper suggests that the new system can process a new user in less than 5 milliseconds, an order of magnitude faster than the current technology. Because of the large number of example depth maps the system is also more versatile across a wider range of body types.

The paper does not make it clear if and when the research will be used on retail Kinect sensor, but as a software solution it could potentially be released as a simple firmware update.

More stories

Phil Spencer "evaluating" Xbox relationship with Activision Blizzard following Kotick allegations

Microsoft's gaming leadership "disturbed and deeply troubled" by claims against Call of Duty publisher's CEO

By James Batchelor

Phil Spencer calls for industry to work on 'legal emulation'

Xbox boss says enabling people to continue playing titles they own "seems like a great North Star" for games companies

By James Batchelor

Latest comments (6)

Kevin Clark-Patterson Lecturer in Games Development, Lancaster and Morecambe College10 years ago
Here's hoping it does make it to retail Kinect's otherwise what is the point?

Once the tracking speed is improved then that will surely lead onto fingers being able to be track and hopefully facial expressions too! :)
0Sign inorRegisterto rate and reply
Neil Young Programmer, Rebellion Developments10 years ago
I thought the lack of finger tracking in the current hardware was due to the resolution of the NIR cameras?

Suspect how soon we see this showing up in games depends on how much data it needs in memory for the depth maps - if they need more memory than the current implementation, this may need new titles / patches for existing titles?
0Sign inorRegisterto rate and reply
John Donnelly Quality Assurance 10 years ago
Kevin, the point is that the technology goes beyond gaming for Microsoft.
Given their R&D budget and work on human computer interfaces this ties in with that high level research that may not always filter down to retail as quickly as some work like.
Look at things like the Microsoft Surface to show what types of R&D is done within Microsoft as a whole.

I do see the fact they have had a team working on this mean there could well be a trend to push the retail technology as far as possible with the same CPU/memory overhead as they currenty need while making the whole thing much more accurate and usable for a wider range of games but at the same time I can see the other side of the business looking to see what developments they can have for the workplace and home markets outside of games.

0Sign inorRegisterto rate and reply
Show all comments (6)
Patrick Frost QA Project Monitor 10 years ago
@Neil - That's what I read as well. They have said that they are working on the data compression so that the USB ports on the X360 can handle the full resolution of the device.

But if I'm honest, I'm really gald that there's progression at the algorithm level. It improves things independently of hardware upgrades and will have knock on benefits.
0Sign inorRegisterto rate and reply
Richard Mitchell Contributing Editor, Joystiq 10 years ago
This paper simply seems to be explaining how the Kinect works. I don't see any references to improving its current performance. The "AI breakthrough" seems to refer to the fact that Kinect handles motion tracking much better than previous attempts.
0Sign inorRegisterto rate and reply
Greg Wilcox Creator, Destroy All Fanboys! 10 years ago

Why not work out these kinks THEN release the product so it WORKS AS IT SHOULD out of the box? I'm a bit tired from being used as an unpaid QA lab rat when these new consoles and peripherals keep landing in stores as "game-changers" when all you're getting is something that kinda, sorta works with some games and kinda, sorta doesn't work with others.

If something goes wrong with them, at least I can bash myself in the head with my Wiimote or Move wand (or pretend I'm the late Lux Interior and sing "Tear it Up" while recalibrating controllers between games). With Kinect, I can only glare at strongly and make faces (and watch it glare back like HAL 9000 before he sends Frank Poole to his death in 2001: A Space Odyssey)...

All that extra exercise I've been getting by sloooowly and carefully flailing my arms an average of seven times to move menus around is only making me head for that heart attack a bit sooner. Thank you Microsoft... now fix this, please...
0Sign inorRegisterto rate and reply

Sign in to contribute

Need an account? Register now.