Title: Detection of dementia on voice recordings using deep learning: a Framingham Heart Study
Abstract Background Identification of reliable, affordable, and easy-to-use strategies for detection of dementia is sorely needed. Digital technologies, such as individual voice recordings, offer an attractive modality to assess cognition but methods that could automatically analyze such data are not readily available. Methods and findings We used 1264 voice recordings of neuropsychological examinations administered to participants from the Framingham Heart Study (FHS), a community-based longitudinal observational study. The recordings were 73 min in duration, on average, and contained at least two speakers (participant and examiner). Of the total voice recordings, 483 were of participants with normal cognition (NC), 451 recordings were of participants with mild cognitive impairment (MCI), and 330 were of participants with dementia (DE). We developed two deep learning models (a two-level long short-term memory (LSTM) network and a convolutional neural network (CNN)), which used the audio recordings to classify if the recording included a participant with only NC or only DE and to differentiate between recordings corresponding to those that had DE from those who did not have DE (i.e., NDE (NC+MCI)). Based on 5-fold cross-validation, the LSTM model achieved a mean (±std) area under the receiver operating characteristic curve (AUC) of 0.740 ± 0.017, mean balanced accuracy of 0.647 ± 0.027, and mean weighted F1 score of 0.596 ± 0.047 in classifying cases with DE from those with NC. The CNN model achieved a mean AUC of 0.805 ± 0.027, mean balanced accuracy of 0.743 ± 0.015, and mean weighted F1 score of 0.742 ± 0.033 in classifying cases with DE from those with NC. For the task related to the classification of participants with DE from NDE, the LSTM model achieved a mean AUC of 0.734 ± 0.014, mean balanced accuracy of 0.675 ± 0.013, and mean weighted F1 score of 0.671 ± 0.015. The CNN model achieved a mean AUC of 0.746 ± 0.021, mean balanced accuracy of 0.652 ± 0.020, and mean weighted F1 score of 0.635 ± 0.031 in classifying cases with DE from those who were NDE. Conclusion This proof-of-concept study demonstrates that automated deep learning-driven processing of audio recordings of neuropsychological testing performed on individuals recruited within a community cohort setting can facilitate dementia screening. more »« less
Automated computational assessment of neuropsychological tests would enable widespread, cost‐effective screening for dementia.
Methods
A novel natural language processing approach is developed and validated to identify different stages of dementia based on automated transcription of digital voice recordings of subjects’ neuropsychological tests conducted by the Framingham Heart Study (n= 1084). Transcribed sentences from the test were encoded into quantitative data and several models were trained and tested using these data and the participants’ demographic characteristics.
Results
Average area under the curve (AUC) on the held‐out test data reached 92.6%, 88.0%, and 74.4% for differentiating Normal cognition from Dementia, Normal or Mild Cognitive Impairment (MCI) from Dementia, and Normal from MCI, respectively.
Discussion
The proposed approach offers a fully automated identification of MCI and dementia based on a recorded neuropsychological test, providing an opportunity to develop a remote screening tool that could be adapted easily to any language.
Zhang, Larry; Ngo, Anthony; Thomas, Jason A.; Burkhardt, Hannah A.; Parsey, Carolyn M.; Au, Rhoda; Hosseini Ghomi, Reza(
, Exploration of Medicine)
null
(Ed.)
Aim:Although clinicians primarily diagnose dementia based on a combination of metrics such as medical history and formal neuropsychological tests, recent work using linguistic analysis of narrative speech to identify dementia has shown promising results. We aim to build upon research by Thomas JA & Burkardt HA et al. (J Alzheimers Dis. 2020;76:905–2) and Alhanai et al. (arXiv:1710.07551v1. 2020) on the Framingham Heart Study (FHS) Cognitive Aging Cohort by 1) demonstrating the predictive capability of linguistic analysis in differentiating cognitively normal from cognitively impaired participants and 2) comparing the performance of the original linguistic features with the performance of expanded features.Methods:Data were derived from a subset of the FHS Cognitive Aging Cohort. We analyzed a sub-selection of 98 participants, which provided 127 unique audio files and clinical observations (n = 127, female = 47%, cognitively impaired = 43%). We built on previous work which extracted original linguistic features from transcribed audio files by extracting expanded features. We used both feature sets to train logistic regression classifiers to distinguish cognitively normal from cognitively impaired participants and compared the predictive power of the original and expanded linguistic feature sets, and participants’ Mini-Mental State Examination (MMSE) scores.Results:Based on the area under the receiver-operator characteristic curve (AUC) of the models, both the original (AUC = 0.882) and expanded (AUC = 0.883) feature sets outperformed MMSE (AUC = 0.870) in classifying cognitively impaired and cognitively normal participants. Although the original and expanded feature sets had similar AUC, the expanded feature set showed better positive and negative predictive value [expanded: positive predictive value (PPV) = 0.738, negative predictive value (NPV) = 0.889; original: PPV = 0.701, NPV = 0.869].Conclusions:Linguistic analysis has been shown to be a potentially powerful tool for clinical use in classifying cognitive impairment. This study expands the work of several others, but further studies into the plausibility of speech analysis in clinical use are vital to ensure the validity of speech analysis for clinical classification of cognitive impairment.
Identifying mild cognitive impairment (MCI) patients at risk for dementia could facilitate early interventions. Using electronic health records (EHRs), we developed a model to predict MCI to all‐cause dementia (ACD) conversion at 5 years.
METHODS
Cox proportional hazards model was used to identify predictors of ACD conversion from EHR data in veterans with MCI. Model performance (area under the receiver operating characteristic curve [AUC] and Brier score) was evaluated on a held‐out data subset.
RESULTS
Of 59,782 MCI patients, 15,420 (25.8%) converted to ACD. The model had good discriminative performance (AUC 0.73 [95% confidence interval (CI) 0.72–0.74]), and calibration (Brier score 0.18 [95% CI 0.17–0.18]). Age, stroke, cerebrovascular disease, myocardial infarction, hypertension, and diabetes were risk factors, while body mass index, alcohol abuse, and sleep apnea were protective factors.
DISCUSSION
EHR‐based prediction model had good performance in identifying 5‐year MCI to ACD conversion and has potential to assist triaging of at‐risk patients.
Highlights
Of 59,782 veterans with mild cognitive impairment (MCI), 15,420 (25.8%) converted to all‐cause dementia within 5 years.
Electronic health record prediction models demonstrated good performance (area under the receiver operating characteristic curve 0.73; Brier 0.18).
Age and vascular‐related morbidities were predictors of dementia conversion.
Synthetic data was comparable to real data in modeling MCI to dementia conversion.
Key Points
An electronic health record–based model using demographic and co‐morbidity data had good performance in identifying veterans who convert from mild cognitive impairment (MCI) to all‐cause dementia (ACD) within 5 years.
Increased age, stroke, cerebrovascular disease, myocardial infarction, hypertension, and diabetes were risk factors for 5‐year conversion from MCI to ACD.
High body mass index, alcohol abuse, and sleep apnea were protective factors for 5‐year conversion from MCI to ACD.
Models using synthetic data, analogs of real patient data that retain the distribution, density, and covariance between variables of real patient data but are not attributable to any specific patient, performed just as well as models using real patient data. This could have significant implications in facilitating widely distributed computing of health‐care data with minimized patient privacy concern that could accelerate scientific discoveries.
Morar, Ulyana; Martin, Harold; M., Robin P.; Izquierdo, Walter; Zarafshan, Elaheh; Forouzannezhad, Parisa; Unger, Elona; Cabrerizo, Mercedes; Curiel Cid, Rosie E.; Rosselli, Monica; et al(
, Cognitive Computation)
Abstract Alzheimer’s disease (AD) is a neurogenerative condition characterized by sharp cognitive decline with no confirmed effective treatment or cure. This makes it critically important to identify the symptoms of Alzheimer’s disease in its early stages before significant cognitive deterioration has taken hold and even before any brain morphology and neuropathology are noticeable. In this study, five different multimodal deep neural networks (MDNN), with different architectures, in search of an optimal model for predicting the cognitive test scores for the Mini-Mental State Examination (MMSE) and the modified Alzheimer’s Disease Assessment Scale (ADAS-CoG13) over a span of 60 months (5 years). The multimodal data utilized to train and test the proposed models were obtained from the Alzheimer’s Disease Neuroimaging Initiative study and includes cerebrospinal fluid (CSF) levels of tau and beta-amyloid, structural measures from magnetic resonance imaging (MRI), functional and metabolic measures from positron emission tomography (PET), and cognitive scores from the neuropsychological tests (Cog). The models developed herein delve into two main issues: (1) application merits of single-task vs. multitask for predicting future cognitive scores and (2) whether time-varying input data are better suited than specific timepoints for optimizing prediction results. This model yields a high of 90.27% (SD = 1.36) prediction accuracy (correlation) at 6 months after the initial visit to a lower 79.91% (SD = 8.84) prediction accuracy at 60 months. The analysis provided is comprehensive as it determines the predictions at all other timepoints and all MDNN models include converters in the CN and MCI groups (CNc, MCIc) and all the unstable groups in the CN and MCI groups (CNun and MCIun) that reverted to CN from MCI and to MCI from AD, so as not to bias the results. The results show that the best performance is achieved by a multimodal combined single-task long short-term memory (LSTM) regressor with an input sequence length of 2 data points (2 visits, 6 months apart) augmented with a pretrained Neural Network Estimator to fill in for the missing values.
Yin, Chenzhong; Imms, Phoebe; Cheng, Mingxi; Amgalan, Anar; Chowdhury, Nahian F.; Massett, Roy J.; Chaudhari, Nikhil N.; Chen, Xinghe; Thompson, Paul M.; Bogdan, Paul; et al(
, Proceedings of the National Academy of Sciences)
The gap between chronological age (CA) and biological brain age, as estimated from magnetic resonance images (MRIs), reflects how individual patterns of neuroanatomic aging deviate from their typical trajectories. MRI-derived brain age (BA) estimates are often obtained using deep learning models that may perform relatively poorly on new data or that lack neuroanatomic interpretability. This study introduces a convolutional neural network (CNN) to estimate BA after training on the MRIs of 4,681 cognitively normal (CN) participants and testing on 1,170 CN participants from an independent sample. BA estimation errors are notably lower than those of previous studies. At both individual and cohort levels, the CNN provides detailed anatomic maps of brain aging patterns that reveal sex dimorphisms and neurocognitive trajectories in adults with mild cognitive impairment (MCI, N = 351) and Alzheimer’s disease (AD, N = 359). In individuals with MCI (54% of whom were diagnosed with dementia within 10.9 y from MRI acquisition), BA is significantly better than CA in capturing dementia symptom severity, functional disability, and executive function. Profiles of sex dimorphism and lateralization in brain aging also map onto patterns of neuroanatomic change that reflect cognitive decline. Significant associations between BA and neurocognitive measures suggest that the proposed framework can map, systematically, the relationship between aging-related neuroanatomy changes in CN individuals and in participants with MCI or AD. Early identification of such neuroanatomy changes can help to screen individuals according to their AD risk.
Xue, Chonghua, Karjadi, Cody, Paschalidis, Ioannis Ch., Au, Rhoda, and Kolachalama, Vijaya B. Detection of dementia on voice recordings using deep learning: a Framingham Heart Study. Retrieved from https://par.nsf.gov/biblio/10303605. Alzheimer's Research & Therapy 13.1 Web. doi:10.1186/s13195-021-00888-3.
Xue, Chonghua, Karjadi, Cody, Paschalidis, Ioannis Ch., Au, Rhoda, & Kolachalama, Vijaya B. Detection of dementia on voice recordings using deep learning: a Framingham Heart Study. Alzheimer's Research & Therapy, 13 (1). Retrieved from https://par.nsf.gov/biblio/10303605. https://doi.org/10.1186/s13195-021-00888-3
Xue, Chonghua, Karjadi, Cody, Paschalidis, Ioannis Ch., Au, Rhoda, and Kolachalama, Vijaya B.
"Detection of dementia on voice recordings using deep learning: a Framingham Heart Study". Alzheimer's Research & Therapy 13 (1). Country unknown/Code not available. https://doi.org/10.1186/s13195-021-00888-3.https://par.nsf.gov/biblio/10303605.
@article{osti_10303605,
place = {Country unknown/Code not available},
title = {Detection of dementia on voice recordings using deep learning: a Framingham Heart Study},
url = {https://par.nsf.gov/biblio/10303605},
DOI = {10.1186/s13195-021-00888-3},
abstractNote = {Abstract Background Identification of reliable, affordable, and easy-to-use strategies for detection of dementia is sorely needed. Digital technologies, such as individual voice recordings, offer an attractive modality to assess cognition but methods that could automatically analyze such data are not readily available. Methods and findings We used 1264 voice recordings of neuropsychological examinations administered to participants from the Framingham Heart Study (FHS), a community-based longitudinal observational study. The recordings were 73 min in duration, on average, and contained at least two speakers (participant and examiner). Of the total voice recordings, 483 were of participants with normal cognition (NC), 451 recordings were of participants with mild cognitive impairment (MCI), and 330 were of participants with dementia (DE). We developed two deep learning models (a two-level long short-term memory (LSTM) network and a convolutional neural network (CNN)), which used the audio recordings to classify if the recording included a participant with only NC or only DE and to differentiate between recordings corresponding to those that had DE from those who did not have DE (i.e., NDE (NC+MCI)). Based on 5-fold cross-validation, the LSTM model achieved a mean (±std) area under the receiver operating characteristic curve (AUC) of 0.740 ± 0.017, mean balanced accuracy of 0.647 ± 0.027, and mean weighted F1 score of 0.596 ± 0.047 in classifying cases with DE from those with NC. The CNN model achieved a mean AUC of 0.805 ± 0.027, mean balanced accuracy of 0.743 ± 0.015, and mean weighted F1 score of 0.742 ± 0.033 in classifying cases with DE from those with NC. For the task related to the classification of participants with DE from NDE, the LSTM model achieved a mean AUC of 0.734 ± 0.014, mean balanced accuracy of 0.675 ± 0.013, and mean weighted F1 score of 0.671 ± 0.015. The CNN model achieved a mean AUC of 0.746 ± 0.021, mean balanced accuracy of 0.652 ± 0.020, and mean weighted F1 score of 0.635 ± 0.031 in classifying cases with DE from those who were NDE. Conclusion This proof-of-concept study demonstrates that automated deep learning-driven processing of audio recordings of neuropsychological testing performed on individuals recruited within a community cohort setting can facilitate dementia screening.},
journal = {Alzheimer's Research & Therapy},
volume = {13},
number = {1},
author = {Xue, Chonghua and Karjadi, Cody and Paschalidis, Ioannis Ch. and Au, Rhoda and Kolachalama, Vijaya B.},
}
Warning: Leaving National Science Foundation Website
You are now leaving the National Science Foundation website to go to a non-government website.
Website:
NSF takes no responsibility for and exercises no control over the views expressed or the accuracy of
the information contained on this site. Also be aware that NSF's privacy policy does not apply to this site.
