Apple

Spatial computing is a fast-growing field that is set to create a new way of interacting with the world around us. This technology has the potential to revolutionize many industries, from healthcare and education to manufacturing and more. The potential is seen by such well-known companies as Apple, Meta, Magic Leap, and others, which work with spatial computing and mixed, augmented, and virtual reality. By 2032, the total value of the spatial computing market will grow to more than $512 billion, compared to $87.5 billion in 2022.

In this article, we will take a closer look at the concept of spatial computing and try to explain how it differs from mixed reality, as well as its potential impact on various industries.

Rise of Spatial Computing. Historical Overview

According to Wikipedia, the term “spatial computing” was first coined in 2003, in the eponymous research paper by Simon Greenwold, a former researcher in the Aesthetics and Computation group at the MIT Media Lab and current head of Story As… Simon Greenwald. The original definition of spatial computing is “a human-machine interaction in which the machine retains and manipulates referents to real objects and spaces”.

With the development of immersive technologies, modern spatial computing primarily describes a direct human interaction with technologies and digital content. For example, spatial computing involves the use of gestures, head and eye movements, voice commands and their integration into the digital interface of a device, such as the latest Apple Vision Pro.

As defined by Apple, spatial computing is a technology that seamlessly blends digital content with the real world. For example, this technology is useful for remote work, games, or movies, where a headset user can expand the screen to gigantic dimensions, making home viewing more like a real cinema experience.

Spatial Computing vs. Mixed Reality. Understanding Key Differences

There is a popular opinion that the term “spatial computing” is not so innovative. According to some experts, this technology works on the principle of mixed reality, popularized by Microsoft with the help of Hololens MR glasses. And indeed, according to Vladimir Grygoriev, Qualium Systems tech lead, both mixed reality and spatial computing impose digital objects on the real space, considering the physics of the space.

“In fact, many devices already do the same spatial mapping as Apple does. And what is Apple planning to present, what is the main innovation and the difference? First, they want to add such image quality that a person will not feel the artificiality of it on devices. Meaning, you have a cup standing in front of you, and when being touched, it falls, spills, and breaks. But when you take off your glasses, it does not even exist,” Grygoriev said.

On the other hand, when releasing the newest headset, Apple plans to use “spatial computing” as a collective term for other areas, such as spatial rendering, spatial mapping, spatial interactions, etc. The innovation of Vision Pro is also about its functionality, with the interface allowing you to turn on several applications at the same time.

“Apple will provide an option, that allows you to run a bunch of applications in small windows in your room. It will also give other people access to your space. You can make 5–10 different screens, where various types of content are being sheared. Then you can close some apps and, wearing a headset, go from the dining room to the kitchen while checking your online chat,” Grygoriev said.

Implementing Spatial Computing and Mixed Reality

Spatial computing is used in such areas as

3D Modeling. Due to the fact that spatial computing technology can seamlessly integrate digital objects into real space, a user of an AR/VR headset can design a 3D model of their future project, whether it is a building, a car, a room, etc. With spatial computing, a user has the ability to zoom in and out a digital object, view it from all sides, etc. This gives the user the opportunity to see the future construction in more detail and correct errors that can cause serious financial losses. For example, in this video, an engineer wearing Hololens 2 glasses projects a future car model on his desk.

Remote collaboration. With spatial computing and mixed reality, employees who are in different places can collaborate and see each other. It can be either a completely virtual environment (for example, a virtual office) or an MR option, in which the employee can see a digital screen with the image of their colleague or his own digital avatar in XR-glasses, without leaving a physical workplace. One of the interesting cases of remote collaboration through MR is the concept of mixed reality from the Airbus airline company. The company’s employees work with customers on the aircraft’s family cabin design. The concept offers a full recreation of the virtual cabin on portable devices, as well as the possibility of collective work of MR-glasses users on a digital model of the cabin superimposed on the real world.

Training and skill development. Mixed reality and spatial computing have found their application for training employees in various fields. The convenience of MR training lies in the fact that the employee acquires practical skills at work with the help of digital instructions that are smoothly superimposed on the workplace. Mixed reality and spatial computing are not only implemented to train doctors and factory workers. For example, mixed reality is used to train military pilots for the F-16, F-18, and F-35 fighter jets. During training from Varjo, the pilot sits in the cockpit with a real control panel. A pilot, while training in MR glasses, sees a real image of himself in the cockpit alongside a virtual environment that reproduces the training ground.

Spatial Computing Technologies. Where and How They’re Making an Impact

Empowering Medical Professionals: Spatial Computing’s Role

Spatial computing is already finding its application in the field of medicine. This technology allows you to create interactive digital twins of patients, so that doctors can examine the body from any angle and see a better picture of their diagnosis, more accurately select drugs and create a treatment plan. Also, digital 3D projections are actively used both during surgical interventions and during training.

For example, the Lynx company, in collaboration with the research center INSERM, created a mixed reality program for training medics and conducting surgeries. Thus, surgeons in MR headsets can visualize a patient’s organ they are going to operate on before and during the procedure itself, calculating all possible risks and consequences. The doctor can also add scans of the patient’s real organs and data from their medical record to provide a more realistic experience. During the operation, the doctor in MR glasses is able to monitor the processes in the patient’s body in real time and manipulate digital content using hand gestures.

How Spatial Computing Enhances Architectural Processes

With spatial computing, architects and designers can create interactive presentations. Such presentations can reassure the client in a more efficient way and show them how the final product will look. In this way, it is possible to present the project more transparently and clearly so that a customer pays attention to the design flaws, which will be eliminated in the future.

The development of architectural projects with the help of spatial computing allows you to take into account the nuances of the future building and calculate possible design errors.

For example, the French real estate firm BNP Paribas Group, together with Magic Leap and Vectuel & RF Studio, developed “the POD”, a special capsule with virtual reality glasses, a client enters into and puts on the glasses. In them, he sees a virtual tour with digital models of the future buildings that can be purchased, as well as a visual tutorial on purchasing real estate.

Spatial Computing’s Role in Modern Manufacturing

In manufacturing, spatial computing is used in several directions, such as virtual training, navigation, remote support, creating a digital double of the supply chain, tracking processes in the chain in real-time, etc. Thanks to spatial computing, an employee receives digital instructions that facilitate his work without leaving the workplace.

The American manufacturing company PBC Linear released an AR training for factory workers using tablets and AR glasses. The main goal of introducing the augmented reality is to reduce the training time of employees from three weeks to three days.

“The first thing we do is we record an operator running through the procedures of the machine. Whether it’s a start-up, tool touch-off, blow-off, or any sort of quality. We’ll have someone standing behind them recording and asking questions. They then take those recordings and those questions up to either their computer to author the instructions, or in front of the machine they’ll put on a HoloLens or Magic Leap and they’ll start creating those instructions, just as the tool maker operator laid out for them. And within just a few hours they’ll have a virtual reality or augmented reality version of that same training,” said Beau Wileman, the Factor of the Future Manager.

Spatial Computing Technologies. Future of Learning in Digital Age

Immersive technologies have been proven to enhance the student’s ability to learn new material and consolidate theoretical skills. According to Arbor XR, students are almost 4 times more emotionally connected to the learning material and 4 times more focused on learning.

With mixed reality and spatial computing, a traditional classroom lesson can be turned into an interesting adventure by visually showing an interactive digital model of the learned subject.

For example, the Titans of Space MR app for the Meta Quest glasses will be useful for those who study astronomy. In this app, the option of virtual tours of the digital universe with interactive models of celestial bodies is available. Students travel on a board of a spaceship and can see, for example, the digital solar system up close.

Spatial Computing Revolutionizing Advertising

In today’s technology era, marketers and advertisers are increasingly paying attention to XR. It is confirmed that with the help of interactive content, customers become more interested in the product offered. Knowing this, 90% of companies already use or plan to introduce mixed reality in their advertising campaigns.

In October 2022, at the NFL games, Gilette created an MR ad for the stadium with a large digital model of the razor.

Spatial computing continues the development of technological innovations that blur the boundaries between the physical and digital worlds. Like virtual, augmented, and mixed reality, spatial computing is a transformative force that will carry on shaping industries for years to come. Advanced devices like Apple Vision Pro offer users a new way of interacting with digital objects that respond to the physics of the real environment. Therefore, the technologies of spatial computing and mixed reality have come in handy in such areas as manufacturing, architecture, mechanical engineering, education, etc.

When presenting the newest AR/VR glasses Vision Pro, Apple positions it as a device for entertainment and remote work through video conferencing and virtual offices. However, there are opinions that Apple Vision Pro can be useful not only for entertainment and games but also, for example, for increasing the productivity of an employee. This means caring for mental health, acquiring new knowledge, and honing old skills.

In this blog post, we will cover the basic concept of productivity and its main components. We will also consider examples from the field of XR, which are aimed at supporting human productivity.

Enhance Productivity through XR Self-Development

By the word “productivity” we usually mean a person’s ability to stay fit, as well as his ability to perform basic work tasks and be creative. The problem of productivity still remains relevant, as in the US only 33% of employees are interested in fulfilling their work duties.

How can you solve the problem of interest and efficiency of your own work? There are several main components that support the productivity of a self-developed person:

Ergonomics of the workplace
Continuous self-development and education
Physical support of the form
Support of psychological health

Organizing workplace with XR

The first step in optimizing your work is improving your workplace. Completing your tasks becomes much faster and more efficient if the workplace is more sophisticated, and the person is isolated from various factors that distract them from work.

According to National Association of Professional Organizers, a poorly designed workplace is a serious problem for employees in various businesses. Yes, 91% of workers said they would be more efficient and productive if their workplaces were better organized.

An optimized workplace will allow a person to do work not only faster, but also better. And in this case, XR offers users several solutions for organizing the workplace:

A virtual office is a fully immersive digital environment in which a person is completely isolated from various distractions. Also, the user of a VR headset can use the functions of creating a comfortable working environment (from a digital copy of the office to more non-standard, fantasy settings) and make the workplace more ergonomic by creating additional digital screens, panels with applications, etc. Such functions are offered by VR-office Immersed.

Mixed reality solutions are another method of improving remote work. This option is suitable for those who aspire to make their workplace more ergonomic but do not want to completely isolate themselves from the outside world. For example, the well-known Meta Quest Pro has the function of overlaying digital screens on an already existing device, like a laptop, for example.

How XR Transforms Self-Education

The next way to improve the efficiency of your work is constant self-education and skills development. A person is also able to keep themselves in shape not only by consolidating old skills but also by learning new knowledge that will improve the quality of work. XR learning courses can be of help through several criteria:

Realism of simulations. With the help of XR-gadgets, a student is able to learn new information in the most realistic environment, reproduced in a digital space;
Emotional connection. Being offered a greater number of opportunities to interact with the content being studied (an opportunity not only to see and hear it but also to interact with digital elements), a person not only gets acquainted with new material but also better realizes and understands it with the help of illustrative examples;
Situational learning. Instead of reading books with theoretical material, a student sees the situation with his own eyes and better understands the learned content.

In addition, learning courses in virtual and augmented reality also increase the employee’s confidence in their knowledge and skills. According to PricewaterhouseCoopers, the level of employees’ confidence in their skills is increased by 275%, and training time on AR/VR devices is reduced by four times.

A good deal of VR/AR training for employees are offered by companies to improve employees’ efficiency and increase their productivity. However, speaking about self-improvement, for example, a freelancer or remote worker can independently choose courses both for general education and comprehensive development. For example, if a person works with foreign partners and has to communicate with them every day, foreign language courses will come in handy.

One such course is Global Academy XR’s Simmula for Meta Quest. These virtual courses are designed to improve the level of English, as well as confidence when speaking a foreign language. During these courses, a student wears a VR headset and immerses themselves in a virtual environment where they communicate with other students using digital avatars. The main goal of these VR English courses is more effective language learning with increased concentration and interest due to the fact that the student collaborates with other people in a digital immersive environment.

Harnessing XR for Physical Well-Being

Many experts recommend constant physical activity for employees, in particular those who lead a sedentary lifestyle. After all, it has been proven that regular exercise has a positive effect not only on well-being and tone but also on mental health. According to the University of Bristol, employees who regularly engaged in physical activity reduced their stress levels by 33% and also improved their ability to cope with stress by 46%.

Thus, remote worker or freelancer can improve their physical and mental well-being with the help of virtual training. Despite the fact that a VR headset user trains in a completely virtual environment, they receive real loads. Immersive digital reality fully reproduces the place of training in which a person engages in physical activity as if they were doing it in a real gym. For example, a person can punch a virtual punching bag, lift a dumbbell, or kick a ball.

“Virtual reality games aren’t always marketed as exercise, but our research shows that when you play them, some can deliver the workout equivalent of walking on a treadmill or cycling on a stationary bike”, said Jimmy Bagley, Ph.D., an associate professor of kinesiology and research director of the Strength and Conditioning Lab at San Francisco State University.

One of the VR apps that Dr. Bagley recommended for training is Supernatural for the Meta Quest headset. This program provides the user with a large selection of training: from boxing to yoga to stretching.

Empowering Mental Health through XR

A key way to keep yourself fit is your mental health work. This also includes the development of stress resistance, prevention of burnout at the workplace, etc. In recent years, burnout at work is a common phenomenon, which is one of the reasons for dismissal from work. 84% of millennials say they have experienced burnout at their current job, and half of millennials surveyed said they left a job due to burnout.

VR therapy is a proven way to prevent burnout. Its main advantage is that a person is distracted from the usual workplace and quickly changes the environment by putting on a virtual reality headset and immersing themselves in the digital space.

Exhale XR | VR Wellness is an example of an VR app to prevent burnout at work and support mental health. With this program, users are distracted from the main irritants at the workplace and are transferred into virtual reality. The main setting resembles an ancient altar, which is located in the middle of a darkened fantasy forest. The app includes not only a direct option for meditation but also other types of activities to calm and support mental health: breathing exercises to relieve stress, improve sleep, and relax. With VR glasses, a user can also walk around the area, see the lake, and engage in other activities: hit a virtual bong, release a firefly from a jar, shoot a bow, etc.

Integrating extended reality into the work routine improves productivity as well as supports mental health and personal development. As it has already been proven, virtual training, meditation, and training courses can effectively increase the productivity of an employee at the workplace and keep them in a resourceful state. If a person is able to independently create an optimal and ergonomic workplace with the help of digital technologies, their performance goes up.

And given the prospects that Apple Vision Pro can provide, the development of virtual, augmented, and mixed reality in the field of productivity support will still go on.

Image: Freepik

Imagine you’re standing at the threshold of Apple’s new XR headset release, eyes keenly following the stock market reactions and your inbox buzzing with LinkedIn updates about Unity’s burgeoning open positions. The anticipation is almost tangible – it’s like the tech industry’s equivalent of awaiting the final season of a blockbuster TV series. But alas, the actual release isn’t due until next year. Does this mean we press pause on developments? Absolutely not!

You might wonder, “Why not take a breather?” The reason is straightforward: MR (Mixed Reality) app development waits for no one. If the groundwork is laid properly now, adapting our apps for the VisionPro release later will be as easy as swapping out smartphone cases. The secret weapon here is experience with the Unity engine. Companies fluent in this technology will navigate VisionPro development with an ease and agility akin to a seasoned marathon runner approaching the home stretch. Having experience with ARKit? That’s a bonus akin to having an extra energy gel in that marathon.

And there’s more! Early access to VisionOS SDK is like getting the keys to a treasure chest. It offers an exclusive chance to study, tinker, and try out the first elements of emulation. It’s an opportunity to dive in and get your hands on the technology of tomorrow, today.

Previous encounters with MR devices like Magic Leap, HTC Vive, Pico 4, and HoloLens 2 also offer invaluable insights. These devices, with their distinct programming languages, offer lessons in MR app development that are as comprehensive as they are diverse. They serve as a practical guide to the symphony of MR tech. When it comes to eye-tracking, gesture control, and voice control, it’s like we’re in a familiar neighborhood. Experience from working on platforms like Meta Oculus and HoloLens 2 instills confidence, despite the anticipation of some subtle differences with the upcoming VisionPro. However, the fundamentals will likely stay the same.

So, to all XR enthusiasts out there, keep those VR goggles firmly in place and maintain the momentum of your MR app developments. While Apple’s new XR headset is an enticing frontier, there’s a lot to be accomplished in the meantime. And when VisionPro does eventually launch, you’ll be primed to embrace it with open arms and innovative apps.

Image: Apple