AI has already begun to be applied in areas where it is necessary to analyze and generate content. In particular, artificial intelligence is implemented in game and app development. AI is getting popular primarily due to its ability to significantly speed up and facilitate the content development process. Instead of spending several hours manually creating graphics for games and applications, you can use AI to generate the necessary components on its own simply. Then the developer will make their edits.
Therefore, it is not surprising that the number of companies using AI is growing. According to Forbes Advisor, by 2027 the artificial intelligence market will be estimated at 407 billion USD. And 64% of enterprises believe that artificial intelligence will increase their productivity.
Unity, one of the most famous app manufacturers for portable gadgets, is among the companies that are already gradually introducing AI into their work.
In this article, we will take a detailed look and analyze the new artificial intelligence developments from Unity and what they mean for the development of XR applications.
Leveraging Unity AI for App Development
Unity has been using AI for a few years now, but, firstly, it was mostly for marketing and non-game content to a wider audience. Like, for example, Luna, a platform that aims to improve the advertising strategy of brands. Or Supersonic which is a stand-alone platform that automates and accelerates game advertising.
Secondly, artificial intelligence and neural networks for gaming have previously been created only for developers who have special qualifications for working with applications. Like, for example, ML agents, which allow you not to code the behavior of digital elements, because these elements themselves (smart agents) are independently learning the algorithms of behavior in the environment. For example, in a game from Unity, you can hone the behavior of a digital corgi that runs after a stick. Or the behavior of racing cars, as in this video.
Read also: Transforming Reality: How AI Revolutionizes XR for Next-Level Experiences
“In the carting micro game, our ML agent scenes are typically set up with a few major components: the training environment, the level, that we want our agents to observe and familiarize with. Which, in our case, is our racetracks. The agents are the game objects we want to train. So, that they can accomplish a goal, such as driving without colliding with the track walls. And, finally — our academy, which collects all observations, done by our agents and trains them”, said Unity’s official video.
In March 2023, Unity announced their own AI game and application development features. Then there was the first presentation of an ecosystem with artificial intelligence, which operates according to the principle of many AI generators, such as Midjourney and ChatGPT.
A few days ago, new AI tools for developing games and applications were announced: Unity Muse and Unity Sentis, which we will discuss in more detail below.
Unity Muse: Empowering AI Creativity
Unity Muse is the newest platform with artificial intelligence, the main purpose of which is to accelerate the creation of applications with 3D graphics. In particular, the option of this platform includes such basic functions as:
- automatic creation of animated characters and digital doubles;
- creation of the movements of the animated model — for example, jumps or backflips;
- creating a texture for the environment;
- addition to ready-made 3D models — for example, the video shows how you can highlight the roof area on the digital model of the house, enter a prompt and select several options for a red roof from the database.
“The Muse platform provides quick access to detailed information from the documentation and can also generate 2D sprites and 3D animations. This allows developers to quickly get the resources they need to create prototypes or new experiences without waiting for final versions of models and sprites. With Muse chat, developers can effectively communicate with other team members and get the help they need during development”, said Qualium Systems Unity tech lead Arcueid D’athemon.
The main difference between Unity Muse and ML agents is ease of use. The newest platform looks like a chat with a neural network and the principle of operation resembles the sensational ChatGPT. As mentioned above, the user enters the required prompt to generate the 3D content required for the application.
Harness the Power of Unity Sentis AI
Unity Sentis is a cross-platform tool that integrates AI into Unity applications. According to Unity’s official announcement, Sentis enables in-app digital models to work flawlessly on all operating systems, browsers, and devices that support Unity, from smartphones and tablets to game consoles and XR glasses.
Thus, applications with built-in AI models work more efficiently due to the fact that data is processed in real-time.
However, according to Arcueid D’athemon, Unity tech lead at Qualium Systems, the range of Sentis options is wider, and the platform can also generate voice acting and animation for a character.
“It opens up opportunities to quickly create dynamically generated actions in the application that will depend on the environment or interaction with the user. With Sentis, developers can effectively create realistic character movements and high-quality voice acting that enriches the user experience”, said Arcueid D’athemon.
In the end, the combination of the input of the own neural network can change the process of the application use. For example, the user can directly interact with an NPC, while would respond to him with lines that are not pre-recorded. It enhances the immersive experience of gameplay, using the example of a virtual AI character named Orb.
Unity’s interest in AI in app development is evident with recent innovations such as Unity Muse and Unity Sentis. The first tool allows developers to quickly and efficiently create 3D models of game and non-game characters, environments, and textures. As for Sentis, this program allows you to modify the gameplay by introducing NPCs with their own artificial intelligence. Unity’s AI ecosystem continues to evolve, inspiring game and app developers to explore new horizons and reimagine what’s possible in interactive entertainment.
Latest Articles
