Nvidia has unveiled an innovative AI-powered platform called "Signs" designed to help people learn American Sign Language (ASL) and bridge communication gaps between deaf and hearing communities.
Developed in partnership with the American Society for Deaf Children and creative agency Hello Monday, Signs uses advanced AI technology combined with a 3D avatar to provide interactive ASL instruction and real-time feedback through webcam analysis.
The platform aims to address a critical need, as ASL ranks as the third most common language in the United States, yet faces a significant shortage of AI-based learning tools compared to English and Spanish.
"Most deaf children are born to hearing parents," explained Cheri Dowling, executive director of the American Society for Deaf Children. "Giving family members accessible tools like Signs enables them to communicate with children as young as six to eight months old."
The platform features a validated library of ASL signs and provides immediate feedback on users' signing technique through AI analysis of webcam footage. While currently focusing on hand movements and finger positions, future versions plan to incorporate facial expressions and head movements - key components of ASL communication.
To build comprehensive resources, Signs allows users of all skill levels to contribute to an open-source video dataset. Nvidia aims to compile 400,000 video clips representing 1,000 signed words, with each entry validated by fluent ASL users and interpreters.
The dataset will be made publicly available later this year, supporting the development of accessible technologies including AI agents, digital human applications, and video conferencing tools.
The team is also working with researchers at Rochester Institute of Technology's Center for Accessibility and Inclusion Research to enhance the platform's user experience for deaf and hard-of-hearing individuals.
The Signs platform represents a major step forward in making ASL education more accessible while helping develop AI applications that can break down communication barriers between deaf and hearing communities.