Platform for medical training in virtual reality

About

V-Med Pro is a VR/MR application designed to revolutionize medical education through interactive 3D anatomy and emergency response scenarios. It enables users to explore the human body and take on various medical roles in a shared virtual space. Supporting up to 20 participants in a single session, the app leverages the Unity Engine and targets the HTC VIVE XR Elite on the Android platform, offering a collaborative learning environment that surpasses traditional training methods.

Challenge

The primary challenge was to create a highly interactive and accurate anatomical model that could be manipulated using both hand gestures and UI while maintaining smooth performance on mobile XR devices. Additionally, it needed to support multi-user sessions for real-time collaboration and scenario management, which required significant optimization

The client approached us to expand their existing AR tablet app into a fully immersive VR/MR application with improved interaction, networked collaboration, and scenario-based learning capabilities, making it a complete educational solution.

Medical training in VR hero

Solution

To address these needs, we divided the project into two phases: Human Body Integration and Scenario Development.

Medical training in VR Solution and scenario development

Phase 1 illustration

Phase 1: Human Body Integration

We began by integrating a detailed human anatomy model, building an intuitive UI for manipulating 990 distinct body parts, and implementing a gesture-based interaction system. It was decided to optimized the graphics using the Universal Render Pipeline (URP) to ensure the application ran smoothly on mobile hardware. Photon Fusion was used to enable real-time collaboration for up to 20 participants, allowing shared interactions within the virtual space. This phase also included a local licensing system for access control and updates.

Phase 1 illustration 1

Phase 1 illustration 2

Phase 1 illustration 3

Phase 1 illustration 4

Phase 1 illustration 5

Phase 1 illustration 6

Phase 2: Scenario Development  and AI Integration

We then developed multiple training scenarios, each tailored to different emergency response roles. Using the OpenAI API, we created AI-driven NPCs that users could interact with through natural language, simulating real-life conversations and responses. It resulted in enhanced immersion and learning, enabling users to practice communication and decision-making in a realistic setting.

Phase 2 illustration

Phase 2 illustration 1

Phase 2 illustration 2

Phase 2 illustration 3

Phase 2 illustration 4

Phase 2 illustration 5

Phase 2 illustration 6

Networking and Performance Optimization

We focused heavily on network stability and real-time synchronization to support multiple users and AI scenarios without performance drops. Photon Fusion enabled smooth interactions and reliable voice communication, ensuring a seamless multi-user experience.

UI/UX Design and Testing

Our team took a user-centric approach to designing the interface, focusing on ease of navigation and functionality. We conducted several iterations with the client’s feedback to refine the UI and interaction models, ensuring that the end product was intuitive and aligned with the educational goals of the platform.

Technologies

Unity

Wave SDK

Open AI

Firebase

Photon

DOTween

Zenject

Team and duration

4 experts

6 months

Social Network Platform

Advancing Neurosurgery with  AR Technology

Let’s discuss your ideas