Scientists have created a new tool that can turn people’s thoughts into words. It works by using an Artificial Intelligence (AI) program to translate brain activity into words. The scientists say the program could help people who can’t speak because of a brain injury.
The new method was developed by a team of scientists at the University of Texas at Austin (UT). The tool has two main parts. The first is a scanner that allowed the scientists to record brain activity. The second is an AI system that the researchers trained to turn videos of brain activity into words.
To record brain activity without surgery, the scientists used a big machine called an fMRI (functional Magnetic Resonance Imaging) machine. The fMRI machine uses strong magnetic fields and radio waves to create pictures showing which parts of the brain are active. The fMRI machine can record the activity of the brain as it happens.
To translate these recordings into words, the researchers used an AI tool called “GPT-1”. Like ChatGPT and similar tools, GPT-1 predicts which word is most likely to come next, based on the information that came before.
The scientists first trained GPT-1 on lots of stories people had told about their lives. Next, the researchers trained GPT-1 to make connections between brain scans and spoken words.
The testing was done on three human volunteers. Each person spent sixteen hours in an fMRI machine listening to stories. The people imagined the stories as they heard them, and the fMRI machine recorded their brain activity.
GPT-1 then used these recordings of brain activity to make connections between the words and ideas in the stories and the brain activity of the listeners.
Then came the test: The researchers played a brand new story. GPT-1 was only given recordings of the people’s brain activity. But the words that GPT-1 predicted were very similar to the words in the story that the people were listening to. The words weren’t exactly the same, but they often carried the same meaning.
For example, when a person was listening to a story about a person who didn’t have a driver’s license, the program came up with this: “She has not even started to learn to drive yet.”
“Our system really works at the level of ideas,” says Alexander Huth, one of the scientists behind the study. “It’s the same idea but expressed in different words.”
The scientists also tested their system when the people imagined their own stories. This didn’t work as well as the stories people heard, but it still worked. Finally, the researchers showed the people silent movies, with no spoken words at all. GPT-1 could still figure out the basic ideas.
The scientists say that GPT-1 is the first AI program to turn what people are thinking into words without brain surgery. The system isn’t something that can be easily used today, mainly because of the size and cost of fMRI machines.
But in the future, a similar method could help people who have lost the ability to speak because of an injury or disease.
Did You Know…?
The scientists say their tool can’t be used to “read people’s minds” without permission. The system only works if the person wants to share their thoughts. Simply thinking about something else will make the system fail. And the training from one person was useless for reading a different person’s thoughts. Even so, the scientists warn that it’s important to think about how similar systems might be misused in the future.