About
Traditional 2D imaging limits neurosurgeons’ ability to fully
visualize and comprehend complex 3D structures. This challenge not
only increases the risk of complications but also prolongs
preoperative planning. !, addresses these issues by overlaying
detailed 3D anatomical models directly onto the physical workspace
in real time.
Challenge
Our team joined the project to turn the existing basic prototype
into a functional high-performance product, addressing the following
challenges:
Application
Performance</span >The development of an AR application to seamlessly process
high-load 3D models and DICOM files.
Connectivity and
Rendering</span >Implement a local WebRTC-based client-server rendering solution
for stable offline processing.
Synchronization and CompatibilityEnsure that headset position, hand gestures, and event tracking
are synchronized across multiple hardware platforms.
Optical Tracking
Reliability</span >Maintain continuous optical tool tracking for precise spatial
synchronization during complex surgical movements.Solution
To overcome the challenges identified and deliver a robust
solution, it was required to develop a series of innovative
approaches that go beyond standard practices. Each solution is
carefully designed to address specific pain points while ensuring
the system remains adaptable and future-proof, laying a strong
foundation for continued innovation and reliability.
Optical Tool Tracking Integration
We leveraged QR code recognition and synchronization mechanisms to
have surgical instruments aligned with spatial coordinates of
Mixed Reality or Spatial Computing glasses and thus enhance
overall operational accuracy.
Real-Time AR 3D Model Alignment
AR 3D model alignment functionality with 3D QR code markers was
incorporated to accurately position virtual models in real-world
environments and ensure continuous tracking.
Local Client-Server Rendering
with WebRTC
To eliminate internet connection dependency, all application
functionalities were transitioned to a desktop-based solution. The
implementation of a WebRTC-based rendering engine allowed us to
stream pixels directly to the Mixed Reality or Spatial Computing
glasses over a local network.
Cross-Platform Hardware Compatibility
We achieved broader compatibility and future adaptability by
decoupling hardware-specific functionalities through the
transmission of positional and event data to a PC.
Technologies
Team and duration
10 experts
4 monthsQualium Systems is a reliable partner in building our Immersive Surgical Platform, both in Augmented Reality and in product development. They show flexibility and expertise in building MedTech product which is always challenging in terms of clinical application, safety and regulation. We were always on the same page in terms of communication and delivery. Thanks to them our startup can move faster and focus on what matters most.
