May 8, 2023
Author: Paalki Sethi
Editor: Dr. Jitendra Kumar Sinha
Modern brain scans can provide highly detailed images of the brain for diagnostic and research purposes. Recently, these scans have been combined with artificial intelligence to translate brain activity into words. It is not surprising that in the coming years, AI produced language will play a significant role in accurately diagnosing brain conditions.
Recent research has shown that brain implants can aid disabled individuals in communication through brain-computer interfaces (BCIs), which primarily focus on the part of the brain responsible for forming words with our mouths. However, a breakthrough technique has been developed that enables the decoding of words and sentences without invasive brain implants, allowing for the reconstruction of continuous language.
This technique relies on artificial intelligence (AI) to translate MRI scans into words and sentences, potentially benefiting patients with brain injuries or paralysis by helping them regain communication abilities. While current BCI approaches utilize methods such as electroencephalogram (EEG) to measure brain activity, they are limited in reconstructing coherent language, having only been effective in deciphering phrases. Originally, BCI technology focused on brain scans for attempting to speak or think about speaking, targeting regions of the brain responsible for movement when attempting to move.
Currently, the core mechanism behind AI interfaces with brain scans remains unclear. However, the basic approach involves reconstructing the essence of what a person hears or imagines, rather than attempting to replicate each word and then decode it. The system requires active cooperation from the participant, which is crucial to ensure non-manipulated results. This factor is vital in any research that involves subjects cooperating to achieve accurate results.
The primary aim of this study is to investigate the fundamental mechanism of language processing in the brain. The use of AI is integral to construct coherent sentences from brain scans. One popular AI program used is Chat GPT, an early version of natural language processing. Initially, the system had difficulties converting brain scans into language. However, researchers incorporated a natural language model using GPT, providing a way to predict what words might follow one another.
To create an interface between the brain and language, computational neuroscientists aim to link specific patterns of brain activity to words, phrases, or sentences. In a recent study, subjects listened to narrative podcasts while their brains were scanned using fMRI. The researchers then created individualized maps illustrating how the brain responds to different words and phrases. Because fMRI records brain activity over a few seconds, it captures the overall idea behind each phrase and sentence rather than just specific words.
The AI used in the study was trained by the team to predict how an individual’s brain would respond to language based on their fMRI data. At first, the device struggled to translate brain images into coherent language. To overcome this, the researchers incorporated the GPT natural language model, which helped predict the sequence of words. They tested different phrases and sentences and checked whether the projected brain activity matched the actual brain activity using the maps and language model. If the match was successful, they moved on to the next word.
The study’s participants were then exposed to podcasts that were not used during the training phase. As the system gradually began to produce words, phrases, and sentences, it generated thoughts that closely resembled what the person was hearing. Although the system occasionally made errors with specific words, it was highly skilled at capturing the essence of the narrative.
Furthermore, the system was effective when the subject watched a silent movie or viewed a video, and the AI attempted to interpret their thoughts. In one experiment, the system correctly generated the phrase “He knocks me to the ground,” while the participant watched an animated film in which a dragon kicked someone. The participants were not instructed to speak during the experiment. Huth noted that this demonstrated that the system is capable of operating at a deeper level than just language.
The brain-computer interface technology has the potential to assist paralyzed individuals who are still conscious but unable to communicate due to brain injuries, strokes, or locked-in syndrome. However, in order to make this a reality, the technology needs to be improved by using more training data and making it more widely accessible. The current device relies on fMRI, which is expensive and difficult to use. Therefore, the team is exploring simpler and more portable imaging techniques like EEG to achieve this goal.
Nevertheless, this groundbreaking technique raises ethical concerns as it could potentially allow individuals to read others’ thoughts, violating their privacy and ethical rights. As a result, researchers must carefully examine the implications of this technology and implement safeguards to prevent its misuse. As the technology continues to develop, it is crucial to prioritize the ethical implications and create safeguards to ensure that the benefits outweigh the risks.
References:
- Surianarayanan, C., Lawrence, J. J., Chelliah, P. R., Prakash, E., & Hewage, C. (2023). Convergence of Artificial Intelligence and Neuroscience towards the Diagnosis of Neurological Disorders-A Scoping Review. Sensors (Basel, Switzerland), 23(6), 3062. https://doi.org/10.3390/s23063062
- Agbavor, F., & Liang, H. (2022). Predicting dementia from spontaneous speech using large language models. PLOS digital health, 1(12), e0000168. https://doi.org/10.1371/journal.pdig.0000168
- Chandler, J. A., Van der Loos, K. I., Boehnke, S., Beaudry, J. S., Buchman, D. Z., & Illes, J. (2022). Brain Computer Interfaces and Communication Disabilities: Ethical, Legal, and Social Aspects of Decoding Speech From the Brain. Frontiers in human neuroscience, 16, 841035. https://doi.org/10.3389/fnhum.2022.841035
- Bohr, A., & Memarzadeh, K. (2020). The rise of artificial intelligence in healthcare applications. Artificial Intelligence in Healthcare, 25–60. https://doi.org/10.1016/B978-0-12-818438-7.00002-2
Recent Articles
.
