Unity's push for "responsibly sourced" AI tools

Sign up for the GI Daily here to get the biggest news straight to your inbox

Unity is pressing on with its push into AI development tools, announcing at today's Unite event that Muse is now in early access.

Despite the debate surrounding the use of artificial intelligence in games development, Unity has been a big proponent for adding AI-powered features into the process, announcing its Muse suite of features earlier this year as well as signing partnerships with AI-driven asset creators. Unveiled earlier this year, Muse is a suite of features designed to assist the engine's users by using artificial intelligence to generate content.

In an interview with GamesIndustry.biz, the president and general manager of Unity Create (the part of the business that handles the engine) Marc Whitten says the company is keen on pursuing AI because it wants to "meet developers where they are, help them, and keep them in control."

"We want them to be able to iterate and experiment faster so they can find the fun in their games and experiences," he says. "This might be asking questions like 'how do I…' that gives information specific to the version of Unity you are using, getting some help optimising some code, creating some variations on a texture that you are playing with, and much more.

"Just like our other tools, we expect to embed our AI tools in the Unity workflow so it's there when you need it if a developer chooses to [use] AI in their workflow."

Muse's first three features include: Muse Chat, which helps developers find resources they're looking for across Unity, including usable code, Muse Sprite, which generates 2D sprites, including multiple variations, and Muse Texture, which creates textures for use in 2D or 3D games.

Other tools on the way include Muse Animate (to create character animations without code), Muse Behaviour (to create character interactions), and Muse Sketch (which Unity describes as a "3D canvas for rapid prototyping and team collaboration").

Unity is also planning to launch Unity Sentis, which allows programmers to embed an AI model with their game or app's Unity Runtime in order to enhance it regardless of which device the player is using. Sentis will debut with Unity 6, formerly Unity 2023 LTS, in 2024.

One of the biggest questions surrounding any AI tool is the datasets it is trained on. There have been scores of examples where AI has generated new images based on stolen content.

Unity already encountered this when it was revealed that Atlas, one of its AI partners, was selling assets that weren't created by AI but stolen from other developers, prompting Unity to cut ties with the company.

Whitten tells us Unity is trying to avoid further issues. For example, Muse Sprite and Muse Texture are "trained entirely on data and images that Unity owns or has licensed."

"To ensure that the outputs are safe and don’t contain any copyrighted materials or recognisable artistic styles, we went through a multi-step process – including both human oversight and machine learning – to generate new synthetic outputs that were re-processed over and over. And then, we applied a custom-built model to this highly processed five million image data set."

Muse Chat, meanwhile, uses a mix of external large language models as well as custom scaffolding systems to understand what is being asked, and produce a response built from Unity's first party resources or others that have been "consensually sourced."

"We are focused on managing our development of AI capabilities in a transparent and responsible manner," Whitten says. "Our approach is to think through how we can deliver tools that are easy for creators to use, responsibly sourced, and with output that creators can feel confident in using in their projects."

Whitten adds that Unity will also announce at Unite that the company will be indemnifying any developers who make use of the Muse tools.

The debate on the use of AI in games development has ramped up in recent years. Whitten emphasises that Unity Muse has been built as a separate product to the engine and that, while he hopes developers and creators try it out, developers can still use Unity the way they always have.

It's easy to see why many are skeptical or worried about the impact of AI on the developer landscape. For example, Muse's tools that create animation and textures might ring alarm bells for animation programmers and texture artists concerned about their roles being replaced.

"We are focused on managing our development of AI capabilities in a transparent and responsible manner"

When asked about how AI might affect the roles available in games development, Whitten says: "We are building AI tools to help creators when they are stuck or want more iterations and give them more tools they can use – if they choose.

"Our goal is to keep the creator in control and in the center of creativity. Just like our other tools like Speedtree, which uses procedural generation to build plants and vegetation, we expect Unity Muse to be another tool in an artist or developer's toolbelt."

Today's Unite event is the company's first major developer gathering since the runtime fee controversy in September, where the engine provider faced backlash over plans to charge studios every time their game was installed after certain milestones. Unity eventually altered these plans, but developers still told GamesIndustry.biz that their trust in the company had been shaken, if not irreparably damaged.

The engine provider is trying to reassure developers it still has their interests at heart, with the runtime fee debacle leading to the departure of CEO John Riccitiello. A keynote speech due to take place later today will see Unity explain how it's "deepening our commitment to game development."

We ask Whitten what Unity is doing to win back developers and their trust, to which he says the firm is "doubling down on the areas that our creators care about the most, specifically the engine," pointing to the news around next year's launch of Unity 6.

He concludes: "We are here to listen and understand our community’s challenges and provide solutions that help."

Sign up for the GI Daily here to get the biggest news straight to your inbox