GiveAHand: Building the world’s largest open-source hand library to help AI unlock Sign Language

Live audio transcription and translation tools have limitations for the deaf and hard-of-hearing due to sign language’s complex combination of fast-paced hand gestures, facial expressions, and full body movements. While machine learning models can handle facial expressions and body movements, detecting hand and finger movements remains a challenge. AI is on the rise and democratising access to data so it can be used in limitless ways. But since most available tools are trained on pre-existing data and images, it’s difficult to build useful machine learning models from sources that aren’t readily available. 

Launched to celebrate the American Sign Language Day (April 15th), GiveAHand.ai is using tech for good. One hundred percent crowdsourced, the data collected in the platform will generate a diverse dataset of hands: diverse shapes, colours, backgrounds and gestures. Now, anyone can put their hands to good use, by contributing and uploading images, helping to build an image library that will help unlock sign language. Researchers can then download and use these fully tagged images to improve their machine learning models, truly allowing the detection and translation of the full spectrum of Sign Language.

“It’s exciting to see the potential of what this platform can unlock in terms of building better, more varied models to serve the Sign Language community. This is a perfect example of how technology can be used to generate progress and inclusion”, said Cheri Dowling, Executive Director of  the American Society for Deaf Children. “We now ask everyone to contribute by – literally – giving us a hand”. 

GiveAHand.ai is the second collaboration between Hello Monday/DEPT® and ASDC: in 2021, they launched Fingerspelling.xyz, a hand tracking experience using machine learning to help learn the sign language alphabet. Currently more than 5.1 million correct hand signs have been registered, and the American Society for Deaf Children now uses Fingerspelling.xyz as part of their own training materials. 

“It’s truly magic when we can combine our love for technology with our desire to create a world where everyone has access to the same language tools. Since Fingerspelling.xyz, and seeing the difference that it makes, we are excited to be build up this dataset and push AI forward in this respect – so that we can help bridge the communication barrier between the deaf and hearing “, said Anders Jessen, CEO and Founder of Hello Monday/DEPT®. 

To contribute and explore the hand image library, you can visit GiveAHand.ai