Technology
An image of a leopard shown to a research subject (left) and an image reconstructed using brain activity using generative AI (right). *Japan Institute of Quantum Science and Technology.
Imagine if artificial intelligence (AI) could read what is going on in the human mind, would you believe it?
A study published this month in the international scientific journal Neural Networks shows that Japanese scientists have used AI technology to successfully create the world's first mental images of objects and landscapes based on human brain activity. This technology, known as "brain decoding," can visualize perceptual content based on brain activity, and is expected to be applied to the medical and welfare fields.
With the rapid development of brain-computer interfaces and neural engineering, it is no longer a science fiction story to use a machine to read out what is going on in the volunteer's brain by analyzing brain activity. The technology of decoding ideas is becoming more and more mature, and the idea of AI "mind reading" is getting closer and closer to reality. At the same time, it has raised concerns about privacy.
Replicate mental imagery.
Previous studies have shown that brain activity measured by functional magnetic resonance imaging (FMRI) reconstructs images seen by humans. But this is limited to specific content such as letters, faces, etc.
Based on the previous approach, a team of scientists from Japan's Institute of Quantum Science and Technology (QST) and Osaka University has developed a technique that quantifies brain activity and combines generative AI and ** technology to draw images to reconstruct complex objects.
The researchers showed participants about 1,200 images of objects and landscapes and used FMRI to analyze and quantify the relationship between their brain signals and visual stimuli, which were then used to train generative AI to decipher and replicate mental imagery from brain activity. As a result, the researchers used this method to produce vivid images, such as leopards with discernible features such as ears, mouths, and spots, as well as objects such as airplanes with red wing lights.
Researchers say the visualization of mental images as arbitrary images of nature is an important milestone.
Humans have used microscopes and other equipment to see the world invisible to the naked eye, but have not yet been able to see one's inner world, the first time humans have peeked into the minds of another person, QST researchers said. Essentially, it can be used to help create new communication devices, while also allowing scientists to explore and understand how hallucinations and dreams work in the brain.
Decode linguistic data.
A new study was published earlier this year in the journal Nature Neuroscience. A research team at the University of Texas at Austin has developed a modern "mind reader" with the help of large language models. They used FMRI to collect information about the brain activity of three participants and convert their thoughts into words with an accuracy rate of 82%.
The blood oxygen level-dependent signal from the fMRI scan shows changes in blood flow and oxygenation levels in different parts of the brain. By focusing on patterns of activity in brain regions and networks that process language, the researchers found that their decoders could be trained to reconstruct continuous language, including the general meaning of some specific words and sentences.
Specifically, the decoder captured the brain responses of the three participants when they listened to the story and generated sequences of words that could produce these brain responses. These word sequences capture the gist of the story well and even include the exact words and phrases. The researchers also asked participants to silence the movie and imagine the movie plot as they scanned their brains. In both cases, the decoder is usually successful in ** the main points of the story and plot. For example, one user thinks "I don't have a driver's license yet" and the decoder says "She hasn't started learning to drive yet". In addition, when a participant actively listens to one story and ignores another story at the same time, the decoder recognizes the meaning of the story that is being actively listened to.
Brain reading technology is a cause for concern.
The idea of technology that can "read minds" raises concerns about psychological privacy. David RodrÃguez Arias Welln, a professor of bioethics at the University of Granada in Spain, warned that this was closer to a future where machines "could read and transcribe thoughts," but that they could go against people's will.
Commenting on research from the University of Texas at Austin, Statnews said that people often describe these types of brain decoders as "mind-readers," but that's a vague term that exaggerates their capabilities. While people's brains produce mental processes, there is limited understanding of how exactly most mental processes are encoded in brain activity.
The article argues that brain decoders cannot simply read out the content of a person's thoughts. Rather, what they learn is to make a mental statement. A brain decoder is like a dictionary that explains between patterns of brain activity and descriptions of mental content. However, brain activity can be affected by factors other than direct stimuli, so the "dictionary" can only ** how a person's brain will respond to stimuli. In addition, it is not feasible to measure a person's brain's response to every possible stimulus. Processes are inherently imperfect, so what a decoder thinks about a person can be very different from what the person actually thinks.
In addition, the article emphasizes that the brain decoder can only "copy" active mental content. All brain recording methods measure signals that correspond to when a person is actively processing information. In contrast, inactive information, such as long-term memory, is encoded in connections between neurons, and people are still far from being able to measure and decode this information.