Piano simulation in Virtual Reality with Passthrough API

September 28, 2021
Piano simulation in Virtual Reality with Passthrough API

Project: Piano simulation on the Oculus Quest 2

The process of learning to play the piano can be quite monotonous and progress is not always clearly visible. This can lead to the student abandoning further training or not putting in enough effort to learn a new skill. This is especially true for children and adolescents.

In order to make the process of learning to play the piano more fun, we decided to use virtual reality technology. With the help of its capabilities, the learning process can be more colorful, exciting and interactive. In order to keep the tactile sensations and sounds from a real piano, we used the PassThrough API. With its help, you can overlay the virtual keyboard on the real one and adjust the number and size of keys. Thus, we are trying to combine the advantages of the virtual and real worlds in order to get a new learning experience.

The application development process continues. Follow our news, there will be many fun and exciting things.

Technology stack: Unity SDK, Oculus MRTK, Oculus Integration.
Devices: Oculus Quest 2.