Here is what the team – composed by Zaiwei Zhang, Long Jiao(AIMove student) and Di Ai (AIMove student) shared with us:
» We only had a basic idea before the hackathon, and during 9 days we succeeded in building a prototype.
We worked on three main axis: the integration of a hand pose estimation framework (MediaPipe), the machine learning algorithms for gesture recognition based on hand skeletons, and the rendering of visual and sound effect into live video stream.
It was a great experience for all three of us, we learned a lot, we discovered the beautiful city Martigny, and we had a great time with other teams and with the organizer Idiap. We even ate a special local raclette served by a robot with kinesthetic teaching from Sylvain Calinon (academic director of AIMove for Idiap)’s team! 🙂
9 days were passed really fast, and we will continue working on this project in order to improve the running speed as well as algorithm accuracy and robustness, and to add more features and UI. »