"So what the hell is Magic Leap doing?"

At the Develop conference today, Graeme Devine described the pitching process that helped Magic Leap discover the true nature of mixed reality content

Magic Leap's Graeme Devine opened his talk at the Develop conference in Brighton this morning with the one thing that nobody in the audience wanted to hear.

"I'm not going to talk about Magic Leap hardware, Magic Leap products, Magic Leap whatever else," he said, to a small but audible sigh of disappointment.

However, Magic Leap's "chief games wizard" didn't lift the veil on exactly what the secretive company is working on, he did offer an insight into the process by which it came to understand what "mixed reality" actually means, and discovered new creative leaders within its ranks. Midway through his talk, Devine posed a question that many in the world of tech and games has no doubt pondered in the last few years: "So what the hell is Magic Leap doing?"

"At Magic Leap, we believe that mixed reality is the platform that will define the world for generations to come"

"At Magic Leap, we believe that mixed reality is the platform that will define the world for generations to come," Devine said. "That it will change humanity, and make humanity better."

First, though, it had to define what Mixed Reality content would actually look like; a process, Devine pointed out, that took media like television and radio decades to refine and perfect. Mixed Reality will go through a similarly protracted evolution, and the evidence that we are in the very early days could even be found within the walls of a bleeding edge company like Magic Leap.

"A lot of people today confuse mixed reality with augmented reality, but mixed reality is completely different," Devine said. Yet that difficulty in drawing a line between MR and AR existed at Magic Leap even three years ago, according to Devine's account, bringing it into line with the company's $542 million series B funding round.

Devine spends a lot of his time exploring the Magic Leap offices, sitting with experts on electronics, lasers, and optics, comic books artists and animators. "All geniuses, every single one," he said. "But the main thing I talked to them about is what the experience will be when they put the device on. Because nobody really buys optics or batteries. They buy the experience... That's the most important thing that Magic Leap makes."

"The company paused for six weeks to work on Pitchfest. We thought it was that important"

Three years ago, Magic Leap created a test application, which represented what the company believed mixed reality to be. It was called General Goes Bananas, a game in which the user fired a bananas out of a gun into a series of baskets, while a four-armed monster tried to intercept and throw them back. "We thought this was awesome," he said. "We thought this was the bees knees and a great demo, and it was cool and fun. But then we slowly realised: this is not mixed reality; this is augmented reality."

AR, Devine said, treats the real world as a surface onto which digital content and information can be projected, but displays no comprehension of the world itself. Magic Leap and mixed reality, on the other hand, use a "sensor set that understands that this is a stage, those are chairs, that's a door, this is a projector, there's a human here and his name is Graeme." To use a slightly pithier description offered by Devine: "Mixed reality is where digital content interacts with the real world and with you."

General Goes Bananas simply laid a digital experience on top of the real world, leading Devine and Magic Leap a hard truth: "We realised that we didn't know what Mixed Reality actually is."

The company "regrouped", and around two years ago Devine launched an initiative inspired by his experiences wandering around Magic Leap: Pitchfest, an event created to gather ideas from the "few hundred" people that worked at the company at that time.

There were a handful of rules: the "five mile test", which is the distance from home a user should be prepared to travel simply to retrieve the application; the "toothbrush test", which asked that the experience be something that could be used every day; the "Halo test", a reference to the Bungie game that convinced people to buy the first Xbox; and the "Innovation test", which asked for experiences that could only exist in mixed reality. As much as anything, though, Devine wanted to see ideas that Magic Leap could learn from, and clarify its own understanding of mixed reality.

Expecting a maximum of 30 responses, the small, internal judging group was "overwhelmed" by almost 200 ideas. The best 20 were picked from that group, and each team was asked to make a long-form presentation from which the judges would select the 10 that would be made into functioning demos. "The company paused for six weeks to work on Pitchfest," Devine said. "We thought it was that important."

"With Pitchfest, we learned a lot. We learned about what Mixed Reality actually is, and more than that we learned about ourselves"

One demo, called Vroom, explored the idea of a racing game with procedurally generated tracks that mapped to the world around the player, who would have to follow the cars as they hurtled under tables and over bookcases and across the ceiling. The combination of the user moving around the space while accurately controlling the cars proved too difficult, though. "I'm not saying we couldn't find a solution here, we just didn't find it with Vroom," Devine said. "But we learned a lot about controls."

The overall winner, though, was Cat-Astrophe, a "crazy cat person simulator" in which the user is surrounded by 20 cats that they must herd into a carrier using tools like a laser pointer and a spray bottle full of water. The cats all interacted independently with the environment and the user, and Magic Leap 3D-printed custom toys and obstacles that could be used to influence their movement and behaviour.

"With Pitchfest, we learned a lot," Devine said. "We learned about what Mixed Reality actually is, and more than that we learned about ourselves. We found new leaders within Magic Leap.

"That process is something we still have inside Magic Leap today. Inside our Interaction Lab we have a process we call 'Experience Incubator'. We still have tons of ideas. We put them through [the Experience Incubator] and we do two- and four-week sprints to see what they're like. Then, at two weeks, we either continue again because they need more time, or we stop and document our learnings... We still do that all the time. That's most of what we do."

Magic Leap's employees are still required to follow the same long-form presentation format devised for Pitchfest, and one of the guiding questions is still: "Why mixed reality?" For Devine, the outcome of this process has provided much needed insight into what the future of mixed reality content will look like, and led to his concept for a 'killer app' called Ghost Girl; a persistent and evolving story that takes place within your home, featuring the spirit of a young girl, which Devine placed into a new category of entertainment experience he calls "Everyday Adventure".

"It could be Star Wars, it could be Harry Potter, it could be Finding Dory, but I will have Everyday Adventure added to my life," he said. "I could be a plumber, I could be a lawyer, I could work at Starbucks, and I will have Everyday Adventure added to my life. I believe Everyday Adventure is the killer app of mixed reality."

The best people to create these experiences, Devine said, are game developers. Earlier in his talk, Devine discussed the potential of mixed reality to "lift people's heads up again" in a world where the "very small four-inch screen" of the smartphone dominates. Mobile apps, however, are primarily good at "hurrying up" to get you specific, personalised information. "They don't really do much beyond that," Devine said. "And when they do it's normally a games programmer that's had the idea - I guarantee it.

"Games applications do interesting things all the time... mixed reality applications will need to do that as well. mixed reality applications need to remain here, to stay with me, to be body-centric, to know the context and the time of day, who I am and who I'm with, all of those things.

"That's a very dynamic kind of application... and the only group of people I know who can make that kind of application are in the games industry. We're about to rule the world."

More stories

Magic Leap appoints Microsoft exec as CEO

Peggy Johnson takes on the role from August 1

By James Batchelor

Magic Leap CEO Ronny Abovitz to step down

Abovitz will continue as CEO until a replacement is found, and remain on the board after the transition

By Matthew Handrahan

Latest comments (4)

"I'm not going to talk about Magic Leap hardware, Magic Leap products, Magic Leap whatever else," he said, to a small but audible sigh of disappointment.
No, it was not an audible sign - it was a audible grown and a slow exit from the hall!

It is time that face the facts that it can save its credibility, or keep favour with the ML guys - but not do both! The whole reason for the invite to DEVELOP was to offer a platform to the team to regain credibility after a number of abortive promised presentations of their technology to prove the rumors of its "snake-oil" status was unfounded.

To then send a representative who played on the fact that he was not prepared to address the serious issues stemming from the recent announced delay and critical reception to the demo for key investors, just adds fuel to what is turning into quite a bombfire of credibility.

If at this coming November presentation ML shows a DK system no better that what MS achieved with HoloLens (two years ago), then I think may need to find a new way to write bad news!
1Sign inorRegisterto rate and reply
Klaus Preisinger Freelance Writing 3 years ago
I am really tired of this PR strategy of not saying anything, but rather tossing trigger words at the consumer trying to spark his imagination in hopes of it running away, turning him into a product evangelist in the process. It got really old over the last five years and when I read something such as Everyday Adventure, or the attempts to differentiate between augmented and mixed reality, I do not mark it down as +1 for team trust, but rather as +1 for team snakeoil. In fact, I say, it was this very PR strategy which gave rise to the snake oil arguments in the first place.

The laws of physics have a way of reigning in magic leaps of any kind, the rest is one stab in the back from Microsoft who are very much in the same business no matter how often Magic Leap will try to claim Microsoft was not.
0Sign inorRegisterto rate and reply
Murray Lorden Game Designer & Developer, MUZBOZ3 years ago
I like the sound of them working on tons of different ideas, to figure out what works well in this new space.
0Sign inorRegisterto rate and reply
Show all comments (4)
Eyal Teler Programmer 3 years ago
The kind of games that Magic Leap talks about is the least appealing to me. I did like Double Fine Action Theater for the Xbox 360 + Kinect a lot. It's great entertainment for little kids, and what Magic Leap talks about falls for me into the same niche. So I feel that it has the same level of appeal as Kinect.

I think that if there's anything transformative about mixed reality games it would be for co-op games, such as board games, darts or paintball. It would be something like the digital transformation of the media marketplace. You could just download a game and play it half-physically, feeling as if it's there, without needing a place to store it, risk losing pieces, making a mess, etc.

Much as I feel that AR (or MR) is the future for computer interaction, a replacement for mobile phones, I feel that for hardcore gaming, VR is where things are.
0Sign inorRegisterto rate and reply

Sign in to contribute

Need an account? Register now.