Yesterday we published the first part of our in-depth discussion with Alex Kipman, Microsoft's director of incubation and creator of Kinect, in which he discussed how Kinect's 'brain' works, how much of the 360's processing power it uses and the parallels between it and a human thinking.
In today's second and final part, we continue our chat with Kipman, touching on what he sees in the future for traditional controllers, what's happening with Star Wars Kinect and why we're not scanning skateboards just yet.
Q:Part of your official Microsoft bio text says that you realised that the 'controller was the greatest single barrier between consumers and gaming'. Is that something that you stand by?
Alex Kipman:More so by the day. I couldn't stand behind that statement more strongly. As I look at technology I find people in my genre of work are busy putting more gadgets and more gizmos in your hand. In this world you're having to take more time to learn technology and you become essentially enslaved to it.
Our world is a world that's trying to turn that on its head. A world that starts to say, 'hey, I can make you the centre of this experience'. I can create technology which fundamentally understands you. I can switch from you having to understand technology to technology understanding all of us.
In that world, the way that you interact with it is a more natural way. I see this happen day after day after day in our playtest labs - when you start using these natural features: voice recognition, full body motion tracking and identity recognition together, the level of immersion and emotional connection you can get with the story deepens, and it deepens in a meaningful way.
That centres on removing this level of abstraction, this level of indirection between you and the story we're trying to tell. Allowing you to have a much more immersive experience. So from that perspective I do think that technology in general and controllers specifically tend to get in the way, and our job as technologists becomes, more and more over time - and this is true not only of Kinect because it doesn't take much to see this monumental shift in the computer industry where conceptual consumer electronics devices are about removing technology and making technology more about understanding you.
Q:Do you mean that you'd rather see the controller disappear completely? Do you not see it as having a function in many game types as a shorthand to a far more complex action?
Alex Kipman:I think that one of the things I appreciate most about Xbox, and why I love the product to pieces, is that it has choice. It allows us a diversified portfolio where we can have content for everyone. Having content for everyone doesn't mean the same cookie cutter answer for every single experience.
It means instead, highly optimised experiences where we have enough breadth to our portfolio that we have content for everyone. This is where I usually like to say '"hey you know what? Xbox was, is, and will continue to be the number one first-person shooter box out there", and those are games that today are traditionally controller games.
We'll continue to invest in these, we'll continue to have controller-only games. We love controller free games, we love Kinect experiences and we'll continue to grow our set of those as well. What we haven't really talked about, but exist, are hybrid games. Games that are using the controller, which we know and love, and pieces, if not all, of the Kinect experiences to again make those experiences more immersive, more fun and more emotionally connected.
This is where I look at the world, and I know it's easier to look at the world and talk about 'or', but I look at the world and I talk about 'and'. It's about how we take all of these things and fuse all of them together to create unique experiences. This is when I go speak and spend time with creative folk around the industry. I go back and I talk about palette. It doesn't always mean using the same colours, the same paintbrushes - the stories you tell are about using the appropriate combinations of all of the colours and brushes to create something meaningful.
So no, I don't see the controller disappearing altogether, but I do see a world which allows all to exist.
Q:Let's talk a little bit about Milo, as I think it represents many of the qualities of Kinect which really grabbed and enthused people. I think it's fair to say that perhaps not too many of the launch titles for Kinect share those inspirational qualities in quite the same way - they're perhaps a little more predictable. Would you like to see Milo, or a project with those qualities, come to market?
Alex Kipman:Well, that's a very nuanced question that I'd love to give you a nuanced answer to and give you the story behind. Milo was never announced as a game. This goes back to me spending a tonne of time working with the creative storytellers in the industry.
Peter Molyneux is probably one of the most amazing people I've had the pleasure of collaborating with. So, there's the world of creating paint colours and paintbrushes - that's me. Then there's the world of creating pictures based on these paint colours and paintbrushes - that's Peter Molyneux, it's a give and take. It's a partnership. Peter comes to me and says, you know what Alex, there are these stories I've always wanted to tell, if only I had these paint colours and paintbrushes. And I say to Peter, 'I have a new selection of paints and paintbrushes - what can you paint with it?'.
You see how these things interchange with each other and then collectively we come up with these transformational, revolutionary experiences. Milo was a sandbox. In this world of creating experiences I used voice, gestures, identity together. Milo was the sandbox which allowed us to define how to do these experiences, and what you saw was a transformational experience where you got a level of emotional connection unlike anything you had seen before.
Now, where has Milo gone? It was never really a product, I will tell you that the technology developed in that sandbox, and by the way we continue to develop technologies in that sandbox, has migrated pretty closely to what you see in a game called Kinectimals.
Kinectimals is about creating an emotional, deep relationship between you and this tiger cub. It uses identity, knows who you are. It actually reacts differently when you walk in front of it, because it's your tiger, than when I walk in front of it, because it doesn't know me. It uses voice, so that you can interact with it and play with it, it uses gestures and essentially moves you to this deep adventure on an island where you're finding the secret of a pirate in much the same way as a traditional adventure type game.
This is one of what I would say was one of the key innovations, that captured people's minds with Milo - this idea that we could create an emotion engine, an engine that would fuse these human input behaviours and create a relationship with this imaginary character. What I think you see in Kinectimals is precisely that.
Q:Do you think it's fair to say that the raft of launch titles is intended as a set of introductory tools and that there'll be a set of deeper experiences later? That these titles are intended to attract a new audiences and that the games for the existing audience are yet to come?
Alex Kipman:The way I would say it is, that November 10 is the beginning of a journey. This is a huge platform release, this is our biggest platform release in the history of Xbox. From that perspective, it is a journey. The fact that we have nineteen titles available for the holiday shows you how much we are putting behind developing a strong portfolio of these experiences. It's our biggest platform launch period. We've never had this many launch titles on any previous launch we've had.
From that perspective, I'll go back and say, we've already shown that we're the number one console for hardcore gamers. We have been, we are and we will continue to be. Part of the explicit design goal with Kinect was to show the range. To show that not only could we be the best hardcore gaming box in the world, but that we could also have other content more suitable for everyone.
It's a managed portfolio for launch, and it's an explicit decision to make sure that all of our content was rated E, for everyone. That's where we start - it's not where we end. You needn't go further than Tokyo Game Show this year where we announced titles that are exclusive to Kinect like Steel Battalion. The next version of that franchise is coming exclusively to our platform. So as our platform moves forward you're going to start seeing the palette, the platform, evolve, much as you saw Xbox Live evolve.
From that perspective you'll continue to see these features showing up in different titles. The titles for launch were explicitly chosen. They were explicitly chosen to be simple, fun and approachable, because look, the line-up for hardcore, we have an amazing line-up as we speak - from Fable to Halo to Black Ops, Gears of War coming up - there's no shortage of hardcore titles coming up, part of Kinect was to show that we had range, that we could start coming up with a more balanced and diversified portfolio. That's where we start. It's by no means where we end.
You should believe that we are going to have more traditional hardcore games that are either hybrid or Kinect exclusive.
Q:I wanted to ask you about something else which we saw at E3, which was the scanning of objects into Kinect, people picking up things in front of the camera and saying, now we're going to put this into the game as an object. We haven't seen that since, is that still something that's possible?
Alex Kipman:It's absolutely possible, you'll see it in the launch titles. I'll give you some examples and then we'll talk about why you're not digitising skateboards.
Kinect Joyride. You can create user generated content from showing the camera real world objects and having the system digitise those objects and translate them into paint colours for your car. I just came from an interview in York city, and I was wearing a t-shirt with a British flag on the sleeve.
Now, I had my tricked out convertible in Kinect Joyride - it was a silver car, but I flashed the sensor my little British flag and the next thing I knew I had this Austin Powers car with red, white and blue stripes across it. That feature ships today in a launch title.
Another example, which goes back to what is probably the most technically challenging, deep title we have in the launch line-up, Kinectimals. With Kinectimals you have these adorable little plush animals that exist in the real world. In the beginning you can go over to this magic stone and flash one of these plushies and that real world plushie shows up in the game as an adorable little tiger cub.
So there's two examples at launch. But why can't you digitise a skateboard? It's easy for me to ideate, on the whiteboard, what I think the world would like. And that's the world of the video, this is where, in this world of creating brand-new, never been seen, transformational experiences, you have to admit that you're stupid. And I'm talking about myself here.
I told my team, during the process, let's not create religion in our stupid phase. This is a point where none of us know how to design games for this world, we need to understand what humans want. We need to observe humans in the playtest lab, we need to see and understand what they like to play. Turns out that, in my stupid phase, I thought scanning skateboards was the single most fun thing known to mankind. Thankfully we didn't create religion during my stupid phase and we started spending time, in the playtests, seeing how people enjoyed digitising real world objects and bringing them to the game.
Turns out, that's cool science fiction, but not fun. From hardcore to mums and dads, casual gamers, they were like 'so why did I do this again'? I spent a good amount of time up front digitising this thing, and it shows up under my guy's feet. Why couldn't I just flash you something, like the colour of my t-shirt, and customise the object that way? And guess what? We listened, and that shows up in games like Kinectimals and Joy Ride.
Now, let's say for some reason that I did want to create a game where I did digitise a skateboard. Could I do it? Sure, absolutely. The platform supports it in terms of giving you the toolset to create these features within your game.
We spent a lot of time learning together as an ecosystem. We had several Kinect summits where we worked with game designers, both first-party and third-party developers, and we shared practices. This was a collective journey where we were all learning together and there was a ton of sharing in terms of what works and doesn't work between first and third-party. Probably unlike anything I've ever seen before.
Turns out not having stuff in your living room and putting you into the action, and using lighter ways of digitising things, is where the fun lies, and that's what you'll see show up in both first and third-party games.
Q:What's happening with the lightsabre game we saw teased at E3? How does Kinect tell which way you're holding an imaginary object like that?
Alex Kipman:Star Wars, unlike Milo, was not a tech demonstration - it was a product. We're really working on it. We haven't disclosed a release date, but we didn't flash it by mistake at E3, you will see a Star Wars game coming soon to Xbox 360 and yes it will be exclusive to Kinect.
In terms of the precision, I will tell you that we have been spending a tremendous amount of time to ensure that this amazing franchise translates in a super-deep and meaningful way. If you're going to create a lightsabre game, that lightsabre game better feel like you always imagined when you dreamed of becoming a Jedi as a kid.
Our level of understanding of the human body transcends precision. It transcends what you can and can't do, and becomes a world where we can create a system that is so science fiction turned science fact that you believe it's magical. This goes beyond how you hold a lightsabre, or how you detect the angle of the lightsabre. That sort of thing we already do in Kinect sports with the ping pong game. You can top-spin, bottom-spin, move it around - we already have the precision in the system to do that.
With a lightsabre you want to be much more intentional than that. You want to feel like the guy's stepping correctly, that the weight of his steps are light - the weight of Jedi steps. You want to feel connected with that. You want to make sure that the lightsabre feels natural and feels connected, when you touch your hands together to get that lightsabre from your pocket, you want to feel that connection of that "wwwhhhhhmmmm" as that thing grows.
I can tell that the answer for how we do it is: very carefully.
Q:We've spoken to Blitz Games and Ubisoft recently and they've told us that they're considering using raw Kinect data instead of processing it through the supplied libraries. What potential advantages could that have?
Alex Kipman:The answer is that you'd have to ask them. I'm the guy who designed the platform - there's a very good reason I gave them access to the raw data. The reason I allow it is the same reason that I allow them to have access to the CPU and GPU at a very core way.
You, as a game designer, choose where you want to go. You can go all the way from middleware, where you have very little control, all the way to flipping zeroes and ones on my GPU and life goes on. Kinect is no different. I provide what I think is a set of sophisticated algorithms that allow you to turn science fiction into science fact. But, I allow everybody to get access to the raw data.
Again, I'm not trying to create religion in my stupid phase. If Ubisoft or anyone else can come along and create more meaningful/cheaper in terms of processing/more optimised content, then God bless them all - they're creating exclusive Kinect content for my platform and I love them to pieces.
Q:I wanted to ask you one last thing about the acquisition of Canesta. Will they be manufacturing more sensors for you or are they going to be working on new tech?
Alex Kipman:Well neither, really. From a manufacturing point of view, the way you can perceive the real world boils down to essentially three ways. One is called structure light, and one is called time of flight and one that's called lines radar. Kinect ships based on structure light. Canesta's technology is based on time of flight. These are separate technologies, so it's not to help with manufacturing, because it's like apples to bananas in terms of how the technologies work.
In terms of future, I will tell you that our focus is an amazing launch and revolutionising gaming and entertainment in the living room. What Microsoft does, and what Microsoft has done in this case, is make an opportunistic buy of a great company that can help us accelerate in meaningful ways, our journey in natural input technology.
I speak generally for Microsoft, not necessarily for Kinect on Xbox.