mind-reading AI hat translates thoughts into text so people can read what others are thinking

brainGPT types thoughts using mind-reading AI hat

Researchers from the GrapheneX-UTS Human-centric Artificial Intelligence Center at the University of Technology Sydney (UTS) have developed a portable and non-invasive mind-reading AI hat called BrainGPT that can decode and translate silent thoughts into readable text. The wearable technology keeps people who are unable to speak due to illness or injury, including stroke or paralysis, in mind when they developed the device, allowing them to communicate with people using data, AI, and a smart device.

video stills courtesy of Thomas Do of Human-centric Artificial Intelligence Centre, via Youtube

The research and development of the mind-reading AI hat were led by Distinguished Professor CT Lin, Director of the GrapheneX-UTS HAI Centre, together with the first author Yiqun Duan and fellow PhD candidate Jinzhou Zhou from the UTS Faculty of Engineering and IT. They tested out their BrainGPT with a group of 29 participants, each of them wearing the mind-reading AI hat while reading passages of text. The wearable technology recorded and translated what they were reading and thinking, using an electroencephalogram (EEG), recording the electrical brain activity through their scalp.

the wearable technology is for people who are unable to speak due to illness or injury, including stroke or paralysis

Non-invasive Wearable technology with AI model

Before they conducted their study with their mind-reading AI hat, the researchers first coded their AI model called DeWave, which helps convert EEG signals into words and sentences. Its data mainly comes from the learned large quantities of EEG data. It is also the foundation of BrainGPT that, unlike Elon Musk’s Neuralink, this wearable technology requires no implantation in the person’s brain. The researchers assure the users that their device is non-invasive and portable and can be used in everyday life.

BrainGPT recorded and translated what the participants were reading and thinking using an EEG

The mind-reading AI hat can also function without additional tools such as eye-tracking. As a standalone wearable technology, BrainGPT can work as it is – the user just needs to put it on and start thinking. Since it is worn as a hat, the researchers note that the signal can be noisier or the user can hear some static. In terms of its performance, the mind-reading AI hat translates and decodes silent thoughts from the participants of the study, but the researchers admit that so far, the current model is more adept at matching verbs than nouns.

the electrical brain activity through the participant’s scalp is converted into words

Improving translation accuracy to 90 percent

Researcher Yiqun Duan says that ‘when it comes to nouns, we saw a tendency towards synonymous pairs rather than precise translations, such as ‘the man’ instead of ‘the author.’ We think this is because when the brain processes these words, semantically similar words might produce similar brain wave patterns. Despite the challenges, our model yields meaningful results, aligning keywords and forming similar sentence structures.’ During the study, the translation accuracy score of BrainGPT was around 40 percent on BLEU-1, and the researchers are working on improving the mind-reading AI hat to a level that is comparable to traditional language translation or speech recognition programs, which is closer to 90 percent.

the researchers first coded their AI model called DeWave, which helps convert the EEG signals into words

As Professor Lin says, their team’s BrainGPT can represent a pioneering effort in translating raw EEG waves directly into language, a significant breakthrough in the field. ‘It is the first to incorporate discrete encoding techniques in the brain-to-text translation process, introducing an innovative approach to neural decoding. The integration with large language models is also opening new frontiers in neuroscience and AI,’ he adds. The study was selected as a spotlight paper at the NeurIPS conference, an annual meeting that showcases world-leading research on artificial intelligence and machine learning, held in New Orleans in 2023.