Revolutionizing Game Development: Unity’s AI Advancements with Unity Muse and Unity Sentis

AI has already begun to be applied in areas where it is necessary to analyze and generate content. In particular, artificial intelligence is implemented in game and app development. AI is getting popular primarily due to its ability to significantly speed up and facilitate the content development process. Instead of spending several hours manually creating graphics for games and applications, you can use AI to generate the necessary components on its own simply. Then the developer will make their edits.

Therefore, it is not surprising that the number of companies using AI is growing. According to  Forbes Advisor, by 2027 the artificial intelligence market will be estimated at 407 billion USD. And 64% of enterprises believe that artificial intelligence will increase their productivity.

Unity, one of the most famous app manufacturers for portable gadgets, is among the companies that are already gradually introducing AI into their work. 

In this article, we will take a detailed look and analyze the new artificial intelligence developments from Unity and what they mean for the development of XR applications.

Leveraging Unity AI for App Development

Unity has been using AI for a few years now, but, firstly, it was mostly for marketing and non-game content to a wider audience. Like, for example, Luna, a platform that aims to improve the advertising strategy of brands. Or Supersonic which is a stand-alone platform that automates and accelerates game advertising.

Secondly, artificial intelligence and neural networks for gaming have previously been created only for developers who have special qualifications for working with applications. Like, for example, ML agents, which allow you not to code the behavior of digital elements, because these elements themselves (smart agents) are independently learning the algorithms of behavior in the environment. For example, in a game from Unity, you can hone the behavior of a digital corgi that runs after a stick. Or the behavior of racing cars, as in this video.

Read also: Transforming Reality: How AI Revolutionizes XR for Next-Level Experiences

“In the carting micro game, our ML agent scenes are typically set up with a few major components: the training environment, the level, that we want our agents to observe and familiarize with. Which, in our case, is our racetracks. The agents are the game objects we want to train. So, that they can accomplish a goal, such as driving without colliding with the track walls. And, finally — our academy, which collects all observations, done by our agents and trains them”, said Unity’s official video.

In March 2023, Unity announced their own AI game and application development features. Then there was the first presentation of an ecosystem with artificial intelligence, which operates according to the principle of many AI generators, such as Midjourney and ChatGPT.

A few days ago, new AI tools for developing games and applications were announced: Unity Muse and Unity Sentis, which we will discuss in more detail below. 

Unity Muse: Empowering AI Creativity

Unity Muse is the newest platform with artificial intelligence, the main purpose of which is to accelerate the creation of applications with 3D graphics. In particular, the option of this platform includes such basic functions as:

  • automatic creation of animated characters and digital doubles;
  • creation of the movements of the animated model — for example, jumps or backflips;
  • creating a texture for the environment;
  • addition to ready-made 3D models — for example, the video shows how you can highlight the roof area on the digital model of the house, enter a prompt and select several options for a red roof from the database.

“The Muse platform provides quick access to detailed information from the documentation and can also generate 2D sprites and 3D animations. This allows developers to quickly get the resources they need to create prototypes or new experiences without waiting for final versions of models and sprites. With Muse chat, developers can effectively communicate with other team members and get the help they need during development”, said Qualium Systems Unity tech lead Arcueid D’athemon.

The main difference between Unity Muse and ML agents is ease of use. The newest platform looks like a chat with a neural network and the principle of operation resembles the sensational ChatGPT. As mentioned above, the user enters the required prompt to generate the 3D content required for the application.

Harness the Power of Unity Sentis AI

Unity Sentis is a cross-platform tool that integrates AI into Unity applications. According to Unity’s official announcement, Sentis enables in-app digital models to work flawlessly on all operating systems, browsers, and devices that support Unity, from smartphones and tablets to game consoles and XR glasses.

Thus, applications with built-in AI models work more efficiently due to the fact that data is processed in real-time.

However, according to Arcueid D’athemon, Unity tech lead at Qualium Systems, the range of  Sentis options is wider, and the platform can also generate voice acting and animation for a character.

“It opens up opportunities to quickly create dynamically generated actions in the application that will depend on the environment or interaction with the user. With Sentis, developers can effectively create realistic character movements and high-quality voice acting that enriches the user experience”, said Arcueid D’athemon.

In the end, the combination of the input of the own neural network can change the process of the application use. For example, the user can directly interact with an NPC, while would respond to him with lines that are not pre-recorded. It enhances the immersive experience of gameplay, using the example of a virtual AI character named Orb.

Unity’s interest in AI in app development is evident with recent innovations such as Unity Muse and Unity Sentis. The first tool allows developers to quickly and efficiently create 3D models of game and non-game characters, environments, and textures. As for Sentis, this program allows you to modify the gameplay by introducing NPCs with their own artificial intelligence. Unity’s AI ecosystem continues to evolve, inspiring game and app developers to explore new horizons and reimagine what’s possible in interactive entertainment.

Latest Articles

How Extended Reality Is Reshaping Modern Marketing
March 31, 2026
How Extended Reality Is Reshaping Modern Marketing

The global extended reality market (including VR, AR and MR) is expected to reach $84.86 billion by 2029, growing at an estimated annual rate of 28%. But the bigger point isn’t just that the market is expanding, it’s that XR is already proving its value in the places marketers care about most: engagement, conversion, and customer confidence. In ecommerce, interacting with products via AR leads to a 94% higher conversion rate compared to products without AR. That makes sense: when people can better understand what they’re buying, they’re more likely to move forward and less likely to regret the purchase later.  XR also gives brands something that’s getting harder to win online: attention. VR campaigns generate about 46% higher engagement than traditional digital campaigns. People who interact with AR content spend around 2.7 times longer on product pages.  XR is now showing up in real results. That is why marketing is moving beyond static content toward immersive experiences. In the following sections, we will share how these technologies can be applied to marketing strategies and explore what the future of immersive experiences might look like. How XR is transforming modern marketing: 4 use cases that prove it works With XR, businesses can turn traditional campaigns into fully immersive experiences, where customers can explore products, interact with brands, and connect with content in memorable ways. Its value goes far beyond visual appeal, directly impacting the business growth and customer journey itself. And while this may not be immediately obvious, XR can also save significant resources, reducing the need for physical prototypes, showrooms, or large-scale events, making marketing more efficient. This is why more businesses are integrating immersive technologies into their marketing strategies, even despite certain challenges, such as development and VR hardware costs, as well as complex technology integration. Below, we highlight several successful use cases of immersive technologies in marketing. Virtual try-ons One of the most persistent barriers to online purchasing is uncertainty. Will these glasses suit my face shape? Will this sofa fit in my living room? Will this shade of lipstick actually complement my skin tone? These are questions that traditionally required a physical store visit. Virtual try-on eliminates that leap entirely. The technology behind this falls into a few distinct forms. The most accessible is smartphone-based AR. Customers point their phone at themselves or their surroundings, and the app overlays a true-to-scale digital product in real time. A striking example is the FindYourGlasses app developed by Qualium Systems. A step further are dedicated AR headsets and glasses, which immerse the customer in a mixed-reality environment where products can be explored in even greater depth and spatial accuracy.  These technologies help customers understand what they are buying before making a purchase, enabling them to make decisions based on accurate, personalized visualization rather than guesswork. Real-world example: IKEA Place AR App IKEA Place AR app lets shoppers visualize furniture in their own physical spaces before buying. Customers simply point their phone camera at a room, select a piece of furniture, and see it rendered in realistic scale within their actual environment. This removes the biggest friction point in furniture shopping: not knowing whether a sofa or shelf will actually fit or match the existing interior design. Results: After launch, the app was downloaded millions of times and became one of the most widely adopted retail AR experiences globally. IKEA reported increased customer engagement and reduced returns because customers could see how items fit before purchase. The company reported also that customers who use the IKEA Place app are 11% more likely to complete a purchase compared to those who do not use the app. Virtual showrooms & Tours Some purchases simply feel too significant to make without experiencing the space or context first. Traditionally, that meant showing up in person. Virtual showrooms and immersive tours remove that requirement. The technology here ranges from 360° web-based tours (viewable in any browser without additional hardware) to fully immersive VR experiences delivered through headsets. Visitors can walk through a branded space, interact with products, and access information on demand, without leaving their couch or office. Automotive brands use virtual showrooms to let buyers explore vehicle interiors, switch trims and colors, and get a feel for the cabin before visiting a dealership. Real estate platforms offer immersive property walkthroughs that let buyers shortlist homes remotely. Hotels and resorts use virtual tours to sell the experience upfront.  The value is especially pronounced in the machinery and heavy equipment sector, where physically demonstrating a product has always been costly: shipping industrial equipment to trade shows, organizing on-site demos, and flying prospects to manufacturing facilities all consume significant budgets. VR removes that overhead entirely: a potential buyer can step inside a virtual factory floor, operate a machine in a simulated environment, and evaluate complex equipment in full detail. Real-world example: Virtual showroom for MAKEEN Energy industrial equipment MAKEEN Energy, a global corporation delivering industrial gas solutions and heavy infrastructure equipment, built a true-to-scale virtual showroom. Using 3D models of their equipment in a virtual environment, they were able to pack their sprawling machinery into a portable VR headset and bring it to any trade fair.  Results: By no longer shipping heavy equipment around the world and reducing travel with virtual product demonstrations, MAKEEN Energy was able to cut logistics costs significantly. The virtual showroom also accelerated complex, multi-stakeholder sales by giving engineers, technicians, and purchase managers across different countries a shared, detailed view of the product. What began as a trade fair tool evolved into a company-wide asset for sales, training, and communications. For industrial businesses looking to adopt XR, Qualium Systems serves as a trusted technology partner, delivering VR and Web3D solutions that simplify the presentation of complex equipment, enhance product understanding, and support more effective digital engagement. Immersive brand storytelling XR gives brands the ability to place customers at the center of a narrative, transforming passive content consumption into a first-person experience that is far harder to forget. A VR film or AR…

June 27, 2025
Methodology of VR/MR/AR and AI Project Estimation

Estimation of IT projects based on VR, XR, MR, or AI requires both a deep technical understanding of advanced technologies and the ability to predict future market tendencies, potential risks, and opportunities. In this document, we aim to thoroughly examine estimation methodologies that allow for the most accurate prediction of project results in such innovative fields as VR/MR/AR and AI by describing unique approaches and strategies developed by Qualium Systems. We strive to cover existing estimation techniques used at our company and delve into the strategies and approaches that ensure high efficiency and accuracy of the estimation process. While focusing on different estimation types, we analyze the choice of methods and alternative approaches available. Due attention is paid to risk assessment being the key element of a successful IT project implementation, especially in such innovative fields as VR/MR/AR and AI. Moreover, the last chapter covers the demo of a project of ours, the Chemistry education app. We will show how the given approaches practically affect the final project estimation. Read

June 27, 2025
What Are Spatial Anchors and Why They Matter

Breaking Down Spatial Anchors in AR/MR Augmented Reality (AR) and Mixed Reality (MR) depend on accurate understanding of the physical environment to create realistic experiences, and they hit this target with the concept of spatial anchors. These anchors act like markers, either geometric or based on features, that help virtual objects stay in the same spot in the real world — even when users move around. Sounds simple, but the way spatial anchors are implemented varies a lot depending on the platform; for example, Apple’s ARKit, Google’s ARCore, and Microsoft’s Azure Spatial Anchors (ASA) all approach them differently. If you want to know how these anchors are used in practical scenarios or what challenges developers often face when working with them, this article dives into these insights too. What Are Spatial Anchors and Why They Matter A spatial anchor is like a marker in the real world, tied to a specific point or group of features. Once you create one, it allows for some important capabilities: Persistence. Virtual objects stay exactly where you placed them in the real-world, even if you close and restart the app. Multi-user synchronization. Multiple devices can share the same anchor, so everyone sees virtual objects aligned to the same physical space. Cross-session continuity. You can leave a space and come back later, and all the virtual elements will still be in the right place. In AR/MR, your device builds a point cloud or feature map by using the camera and built-in sensors like the IMU (inertial measurement unit). Spatial anchors are then tied to those features, and without them, virtual objects can drift or float around as you move, shattering the sense of immersion. Technical Mechanics of Spatial Anchors At a high level, creating and using spatial anchors involves a series of steps: Feature Detection & Mapping To start, the device needs to understand its surroundings: it scans the environment to identify stable visual features (e.g., corners, edges). Over time, these features are triangulated, forming a sparse map or mesh of the space. This feature map is what the system relies on to anchor virtual objects. Anchor Creation Next, anchors are placed at specific 3D locations in the environment in two possible ways: Hit-testing. The system casts a virtual ray from a camera to a user-tapped point, then drops an anchor on the detected surface. Manual placement. Sometimes, developers need precise control, so they manually specify the exact location of an anchor using known coordinates, like ensuring it perfectly fits on the floor or another predefined plane. Persistence & Serialization Anchors aren’t temporary — they can persist, and here’s how systems make that possible: Locally stored anchors. Frameworks save the anchor’s data, like feature descriptors and transforms, in a package called a “world map” or “anchor payload”. Cloud-based anchors. Cloud services like Azure Spatial Anchors (ASA) upload this anchor data to a remote server to let the same anchor be accessed across multiple devices. Synchronization & Restoration When you’re reopening the app or accessing the anchor on a different device, the system uses the saved data to restore the anchor’s location. It compares stored feature descriptors to what the camera sees in real time, and if there’s a good enough match, the system confidently snaps the anchor into position, and your virtual content shows up right where it’s supposed to. However, using spatial anchors isn’t perfect, like using any other technology, and there are some tricky issues to figure out: Low latency. Matching saved data to real-time visuals has to be quick; otherwise, the user experience feels clunky. Robustness in feature-scarce environments. Blank walls or textureless areas don’t give the system much to work with and make tracking tougher. Scale drift. Little errors in the system’s tracking add up over time to big discrepancies. When everything falls into place and the challenges are handled well, spatial anchors make augmented and virtual reality experiences feel seamless and truly real. ARKit’s Spatial Anchors (Apple) Apple’s ARKit, rolled out with iOS 11, brought powerful features to developers working on AR apps, and one of them is spatial anchoring, which allows virtual objects to stay fixed in the real world as if they belong there. To do this, ARKit provides two main APIs that developers rely on to achieve anchor-based persistence. ARAnchor & ARPlaneAnchor The simplest kind of anchor in ARKit is the ARAnchor, which represents a single 3D point in the real-world environment and acts as a kind of “pin” in space that ARKit can track. Building on this, ARPlaneAnchor identifies flat surfaces like tables, floors, and walls, allowing developers to tie virtual objects to these surfaces. ARWorldMap ARWorldMap makes ARKit robust for persistence and acts as a snapshot of the environment being tracked by ARKit. It captures the current session, including all detected anchors and their surrounding feature points, into a compact file. There are a few constraints developers need to keep in mind: World maps are iOS-only, which means they cannot be shared directly with Android. There must be enough overlapping features between the saved environment and the current physical space, and textured structures are especially valuable for this, as they help ARKit identify key points for alignment. Large world maps, especially those with many anchors or detailed environments, can be slow to serialize and deserialize, causing higher application latency when loading or saving. ARKit anchors are ideal for single-user persistence, but sharing AR experiences across multiple devices poses additional issues, and developers often employ custom server logic (uploading ARWorldMap data to a backend), enabling users to download and use the same map. However, this approach comes with caveats: it requires extra development work and doesn’t offer native support for sharing across platforms like iOS and Android. ARCore’s Spatial Anchors (Google) Google’s ARCore is a solid toolkit for building AR apps, and one of its best features is how it handles spatial anchors: Anchors & Hit-Testing ARCore offers two ways to create anchors. You can use Session.createAnchor(Pose) if you already know the anchor’s position, or…



Let's discuss your ideas

Contact us