"So what the hell is Magic Leap doing?"

Magic Leap's Graeme Devine opened his talk at the Develop conference in Brighton this morning with the one thing that nobody in the audience wanted to hear.

"I'm not going to talk about Magic Leap hardware, Magic Leap products, Magic Leap whatever else," he said, to a small but audible sigh of disappointment.

However, Magic Leap's "chief games wizard" didn't lift the veil on exactly what the secretive company is working on, he did offer an insight into the process by which it came to understand what "mixed reality" actually means, and discovered new creative leaders within its ranks. Midway through his talk, Devine posed a question that many in the world of tech and games has no doubt pondered in the last few years: "So what the hell is Magic Leap doing?"

"At Magic Leap, we believe that mixed reality is the platform that will define the world for generations to come"

"At Magic Leap, we believe that mixed reality is the platform that will define the world for generations to come," Devine said. "That it will change humanity, and make humanity better."

First, though, it had to define what Mixed Reality content would actually look like; a process, Devine pointed out, that took media like television and radio decades to refine and perfect. Mixed Reality will go through a similarly protracted evolution, and the evidence that we are in the very early days could even be found within the walls of a bleeding edge company like Magic Leap.

"A lot of people today confuse mixed reality with augmented reality, but mixed reality is completely different," Devine said. Yet that difficulty in drawing a line between MR and AR existed at Magic Leap even three years ago, according to Devine's account, bringing it into line with the company's $542 million series B funding round.

Devine spends a lot of his time exploring the Magic Leap offices, sitting with experts on electronics, lasers, and optics, comic books artists and animators. "All geniuses, every single one," he said. "But the main thing I talked to them about is what the experience will be when they put the device on. Because nobody really buys optics or batteries. They buy the experience... That's the most important thing that Magic Leap makes."

"The company paused for six weeks to work on Pitchfest. We thought it was that important"

Three years ago, Magic Leap created a test application, which represented what the company believed mixed reality to be. It was called General Goes Bananas, a game in which the user fired a bananas out of a gun into a series of baskets, while a four-armed monster tried to intercept and throw them back. "We thought this was awesome," he said. "We thought this was the bees knees and a great demo, and it was cool and fun. But then we slowly realised: this is not mixed reality; this is augmented reality."

AR, Devine said, treats the real world as a surface onto which digital content and information can be projected, but displays no comprehension of the world itself. Magic Leap and mixed reality, on the other hand, use a "sensor set that understands that this is a stage, those are chairs, that's a door, this is a projector, there's a human here and his name is Graeme." To use a slightly pithier description offered by Devine: "Mixed reality is where digital content interacts with the real world and with you."

General Goes Bananas simply laid a digital experience on top of the real world, leading Devine and Magic Leap a hard truth: "We realised that we didn't know what Mixed Reality actually is."

The company "regrouped", and around two years ago Devine launched an initiative inspired by his experiences wandering around Magic Leap: Pitchfest, an event created to gather ideas from the "few hundred" people that worked at the company at that time.

There were a handful of rules: the "five mile test", which is the distance from home a user should be prepared to travel simply to retrieve the application; the "toothbrush test", which asked that the experience be something that could be used every day; the "Halo test", a reference to the Bungie game that convinced people to buy the first Xbox; and the "Innovation test", which asked for experiences that could only exist in mixed reality. As much as anything, though, Devine wanted to see ideas that Magic Leap could learn from, and clarify its own understanding of mixed reality.

Expecting a maximum of 30 responses, the small, internal judging group was "overwhelmed" by almost 200 ideas. The best 20 were picked from that group, and each team was asked to make a long-form presentation from which the judges would select the 10 that would be made into functioning demos. "The company paused for six weeks to work on Pitchfest," Devine said. "We thought it was that important."

"With Pitchfest, we learned a lot. We learned about what Mixed Reality actually is, and more than that we learned about ourselves"

One demo, called Vroom, explored the idea of a racing game with procedurally generated tracks that mapped to the world around the player, who would have to follow the cars as they hurtled under tables and over bookcases and across the ceiling. The combination of the user moving around the space while accurately controlling the cars proved too difficult, though. "I'm not saying we couldn't find a solution here, we just didn't find it with Vroom," Devine said. "But we learned a lot about controls."

The overall winner, though, was Cat-Astrophe, a "crazy cat person simulator" in which the user is surrounded by 20 cats that they must herd into a carrier using tools like a laser pointer and a spray bottle full of water. The cats all interacted independently with the environment and the user, and Magic Leap 3D-printed custom toys and obstacles that could be used to influence their movement and behaviour.

"With Pitchfest, we learned a lot," Devine said. "We learned about what Mixed Reality actually is, and more than that we learned about ourselves. We found new leaders within Magic Leap.

"That process is something we still have inside Magic Leap today. Inside our Interaction Lab we have a process we call 'Experience Incubator'. We still have tons of ideas. We put them through [the Experience Incubator] and we do two- and four-week sprints to see what they're like. Then, at two weeks, we either continue again because they need more time, or we stop and document our learnings... We still do that all the time. That's most of what we do."

Magic Leap's employees are still required to follow the same long-form presentation format devised for Pitchfest, and one of the guiding questions is still: "Why mixed reality?" For Devine, the outcome of this process has provided much needed insight into what the future of mixed reality content will look like, and led to his concept for a 'killer app' called Ghost Girl; a persistent and evolving story that takes place within your home, featuring the spirit of a young girl, which Devine placed into a new category of entertainment experience he calls "Everyday Adventure".

"It could be Star Wars, it could be Harry Potter, it could be Finding Dory, but I will have Everyday Adventure added to my life," he said. "I could be a plumber, I could be a lawyer, I could work at Starbucks, and I will have Everyday Adventure added to my life. I believe Everyday Adventure is the killer app of mixed reality."

The best people to create these experiences, Devine said, are game developers. Earlier in his talk, Devine discussed the potential of mixed reality to "lift people's heads up again" in a world where the "very small four-inch screen" of the smartphone dominates. Mobile apps, however, are primarily good at "hurrying up" to get you specific, personalised information. "They don't really do much beyond that," Devine said. "And when they do it's normally a games programmer that's had the idea - I guarantee it.

"Games applications do interesting things all the time... mixed reality applications will need to do that as well. mixed reality applications need to remain here, to stay with me, to be body-centric, to know the context and the time of day, who I am and who I'm with, all of those things.

"That's a very dynamic kind of application... and the only group of people I know who can make that kind of application are in the games industry. We're about to rule the world."