The Idea
FindYourGlasses APP is an application allowing users to try on the
glasses from the collection before buying them. Additionally, the
app gives recommendations of glasses models based on the user’s face
shape and user personal preferences.
The Challenge
The main difficulty in the implementation of this project was that
it was necessary to accurately determine the shape of the head in
order to give relevant recommendations on the selection of the shape
of the glasses. It was also necessary to implement a clear
positioning of the glasses on the face when the user moves their
head.
The solution
The problem of multiple existing apps is their low performance and a very limited user experience. We created our solution based on cutting-edge technologies allowing users to try on different glasses models in real-time (opposite to the apps providing static images). Our face tracking module allows glasses to stay exactly where they should be when the user moves their head.
Additionally, we have a smart helper feature that can recommend users certain models based on their face shape. In order to make this possible, we studied the patterns that underlie the proportional division of parts of the head. Knowledge of the classical canons gave us the opportunity to correctly determine the proportions and characteristic features of the structure of the heads of different people.
We have developed a custom algorithm, which creates and predicts the whole face key points and distance between them allowing the system to determine the shape of the face with 95.8% accuracy. It is based on the face geometry rules and proportions used by artists.
This is an important part as standard recognition and face tracking algorithms don’t track the forehead points, and without them, it is impossible to correctly determine the shape of the face and provide matching glasses models.
Technologies
C Sharp
Unity SDK
AR Kit
AR Core
Tensor Flow