(GPT-3 algorithm has shown 80% accuracy in predicting the early stages of dementia from spontaneous speech)
January 30, 2023
Author: Manish Verma
Editor: Dr. Jitendra Kumar Sinha
For years, the only tools we had for checking blood pressure and body temperature were arm cuffs and thermometers. These metrics are now possible with wearable tech, including smartwatches. Similarly, neurologists have traditionally depended on invasive and expensive procedures to identify the existence of neurological illnesses, such as lumbar punctures and positron emission tomography scans, but new digital technologies may provide a whole new paradigm for detecting, diagnosing, and monitoring these diseases. With the development of technology such as artificial intelligence, machine learning and deep learning, researchers are working harder than ever to identify digital biomarkers that may be used to spot illnesses in their earliest stages. Digital biomarkers (physiological data collected digitally using devices that can be utilized as a biomarker to identify disorders) have the potential to improve current health assessments as well as pave the way for future measurements. They might make it possible for healthcare to transition from a preventative to a reactive model. For example, a recent study predicted Alzheimer’s using language impairment as a biomarker using chat GPT-3 from spontaneous speech. Let’s take a look.
GPT-3 has recently received a lot of attention from people all across the world due to its advanced functionality. Chat GPT 3 has several capabilities that Google does not. For example, Google can’t write a play Chat GPT 3 on the other hand can.
GPT-3 (Generative Pretrained Transformer 3) is a state-of-the-art language processing AI model developed by OpenAI. It is capable of generating human-like text and has a wide range of applications, including language translation, language modelling, and generating text for applications such as chatbots. It is one of the largest and most powerful language processing AI models to date, with 175 billion parameters.
In a recent peer-reviewed study, OpenAI’s GPT-3 algorithm was found to be 80% accurate in predicting the early stages of dementia from spontaneous speech.
Alzheimer’s disease is currently diagnosed by a medical history review and a battery of physical and neurological assessments and testing. While there is still no cure for the illness, early detection can provide patients with additional treatment and support choices. Because language impairment is a symptom in 60-80% of dementia patients, researchers have focused on systems that can detect minor signs, such as hesitation, grammar and pronunciation errors, and forgetting the meaning of words, as a short test that might suggest whether or not a patient should undergo a complete evaluation.
The Drexel study, published in the journal PLOS Digital Health, is the most recent in a line of initiatives to demonstrate the efficacy of natural language processing algorithms for Alzheimer’s early detection by drawing on recent findings that imply that language impairment can serve as a warning sign for neurodegenerative diseases.
The researchers put their hypothesis to the test by training the software with transcripts from a subset of a dataset of voice recordings produced expressly to assess natural language processing programs’ capacity to detect dementia. The software produced a “embedding”—a distinctive profile of Alzheimer’s speech—by extracting significant elements of the text’s word choice, sentence construction, and meaning.
A. The acoustic features are engineered to capture the pathological speech behavior as well as the acoustic elements of speech.
B. The linguistic features are taken from the text that has been transcribed and are represented as text embeddings.
Fig: Diagram demonstrating two distinctive feature representations that are taken from speech.
(directly reproduced from Felix et al., 2022)
The embedding was then utilized to retrain the algorithm, transforming it into an Alzheimer’s screening machine. To test it, researchers gave the software access to the dataset and prompted it to evaluate hundreds of transcripts and determine whether or not each one was written by an individual who was developing Alzheimer’s.
The team tested two of the best natural language processing algorithms side by side and discovered that GPT-3 outperformed both in terms of accurately identifying Alzheimer’s and non-Alzheimer’s subjects, with fewer missing cases by both the programs.
A second test utilized GPT-3’s textual analysis to forecast patients’ scores on the Mini-Mental State Exam, a widely used assessment for determining the severity of dementia (MMSE).
The researchers next compared the GPT-3’s prediction accuracy against a method that predicted the MMSE score only based on the acoustic characteristics of the recordings, such as voice strength, pauses, and slurring. In terms of predicting patients’ MMSE scores, GPT-3 was about 20% more accurate.
The results revealed that the text embedding created by GPT-3 may be used to reliably differentiate patients with Alzheimer’s disease from healthy controls, as well as predict the subject’s cognitive assessment score, both based purely on voice data. The study’s findings also shown that text embedding outperforms the traditional acoustic feature-based method and even performs competitively with tuned models. Together, the results suggest that GPT-3-based text embedding is a viable method for assessing AD and may help with the early detection of dementia.
Additionally, the researchers intend to create a web application that may be used as a pre-screening tool at home or in a doctor’s office in order to expand on these encouraging results.
However, there is still a need to validate the efficacy of these technologies on a wide scale before using them to diagnose the conditions in clinical settings.
References:
- Agbavor F, Liang H (2022) Predicting dementia from spontaneous speech using large language models. PLOS Digit Health 1(12): e0000168. https://doi.org/10.1371/journal.pdig.0000168
- Balagopalan, A., Eyre, B., Robin, J., Rudzicz, F., & Novikova, J. (2021). Comparing Pre-trained and Feature-Based Models for Prediction of Alzheimer’s Disease Based on Speech. Frontiers in aging neuroscience, 13, 635945. https://doi.org/10.3389/fnagi.2021.635945
- Sezgin, E., Sirrianni, J., & Linwood, S. L. (2022). Operationalizing and Implementing Pretrained, Large Artificial Intelligence Linguistic Models in the US Health Care System: Outlook of Generative Pretrained Transformer 3 (GPT-3) as a Service Model. JMIR medical informatics, 10(2), e32875. https://doi.org/10.2196/32875
- Ghosh, S., Durgvanshi, S., Agarwal, S., Raghunath, M., & Sinha, J. K. (2020). Current Status of Drug Targets and Emerging Therapeutic Strategies in the Management of Alzheimer’s Disease. Current neuropharmacology, 18(9), 883–903. https://doi.org/10.2174/1570159X18666200429011823