Sound processing for NFT project

June 1, 2022
Sound processing for NFT project

We are working on an interesting NFT project related to 3D avatars, video and sound recording. We decided to briefly tell you about the solutions that we use for work, as well as share information about the inner workings of the project.

About the essence of the NFT project

The essence of the NFT project is that you buy unique 3D avatars in the application. For them, you can record video with sound using the front camera of your smartphone. The main feature is that the avatar pronounces the recorded sound, repeating the facial expressions and emotions of the speaker.

There are currently more than 2000 avatars in the application. The application is currently only available for iPhone.

In the future, the purchased NFT avatar can be upgraded in terms of popularity and recognition. After that, it can be monetized: used for advertising, taking part in the shooting of cartoons, animated videos, etc.

About the complexities

While working on the project, it became necessary to impose various effects on the recorded sound. This can enhance the authenticity of the avatar, make the video more impactful with interesting sound effects, and diversify the experience that users can implement in the application.

The main requirement was to use the effects used by professional audio editors. We started looking for a suitable solution, offered various options, some of which did not fit for objective reasons, and the other part was not compatible with the mobile platform. In the end, we settled on one well-known tool that met all our requirements.

Tools for working with sound and video

Since our project needs to record videos with sound and work with them at the same time, we chose FFMpeg – a complete, cross-platform solution to record, convert and stream audio and video. Before that, we used this library for video processing: trimming, inserting background music.

They also knew about the possibilities of working with sound, but did not use all the advanced features. And the possibilities are wide. We can say that this is the most complete library for audio and video processing: compression, timbre, frequency, playback speed, echo. Over 100 different filter combinations.

For future clients

If you have ideas that you would like to implement, please contact us. We will conduct research, choose the best options and make an effective and simple solution.