From american sign language to texts using AI ring

 

Researchers led by Cornell University develop a wearable AI ring that converts American Sign Language into texts for smartphones and computers. Using micro-sonar technology, the device can continuously and in real time track fingerspelling in American Sign Language. So far, users can use the device named ‘SpellRing’ to enter texts into other devices like smartphones and computers with fingerspelling. It’s the sign language method used to spell out words without corresponding signs, including proper nouns, names, and technical terms.

 

Users put the wearable AI ring that converts sign language on their thumb. The device comes with a microphone and speaker. Together, they send and receive inaudible sound waves that track the wearer’s hand and finger movements. Then, there’s also a mini-gyroscope that tracks the hand’s motion. All of these are inside a 3D-printed ring and casing ‘no bigger than a standard U.S. quarter,’ the researchers describe. After that, the deep-learning algorithm processes the sonar images and predicts the American Sign Language fingerspelled letters in real time.

AI ring sign language
all images courtesy of the researchers from Cornell University, Carnegie Mellon University, Stony Brook University

 

 

accuracy rate of texts is between 82 and 92 percent

 

At the present time, the researchers from Cornell University, Carnegie Mellon University, and Stony Brook University have tested the wearable AI ring that converts American Sign Language into texts. The team has asked 20 experienced and novice ASL signers to finger spell a total of more than 20,000 words of varying lengths. They add that the accuracy rate of the translated texts is between 82 and 92 percent, depending on the difficulty of words.

 

From here, the developers continue to train the AI system to recognize 26 handshapes associated with each letter of the alphabet. It’s not easy to capture, all as the variations between letters significantly differ, but the team pushes forward with their research. With further development, the wearable AI ring, which the researchers believe to be the first of its kind, can help with the translation of American Sign Language by continuously tracking entire signed words and sentences. So far, the device can convert ASL. There’s no news yet on its commercial availability.

AI ring sign language
the wearable AI ring that converts American Sign Language into texts for smartphones and computers

AI ring sign language
the device comes with a microphone and speaker to send and receive inaudible sound waves

then, there’s also a mini-gyroscope that tracks the hand’s motion
then, there’s also a mini-gyroscope that tracks the hand’s motion

the components are inside a 3D-printed ring and casing ‘no bigger than a standard U.S. quarter'
the components are inside a 3D-printed ring and casing ‘no bigger than a standard U.S. quarter’

 

 

project info:

 

name: SpellRing

institutions: Cornell University, Carnegie Mellon University, Stony Brook University | @cornelluniversity, @carnegiemellon, @stonybrooku

researchers: Hyunchul Lim, Nam Anh Dang, Dylan Lee, Tianhong Catherine Yu, Jane Lu, Franklin Mingzhe Li, Yiqi Jin, Yan Ma, Xiaojun Bi, François Guimbretière, Cheng Zhang