skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Multimodal Fusion of Smart Home and Text-based Behavior Markers for Clinical Assessment Prediction
New modes of technology are offering unprecedented opportunities to unobtrusively collect data about people's behavior. While there are many use cases for such information, we explore its utility for predicting multiple clinical assessment scores. Because clinical assessments are typically used as screening tools for impairment and disease, such as mild cognitive impairment (MCI), automatically mapping behavioral data to assessment scores can help detect changes in health and behavior across time. In this article, we aim to extract behavior markers from two modalities, a smart home environment and a custom digital memory notebook app, for mapping to 10 clinical assessments that are relevant for monitoring MCI onset and changes in cognitive health. Smart-home-based behavior markers reflect hourly, daily, and weekly activity patterns, while app-based behavior markers reflect app usage and writing content/style derived from free-form journal entries. We describe machine learning techniques for fusing these multimodal behavior markers and utilizing joint prediction. We evaluate our approach using three regression algorithms and data from 14 participants with MCI living in a smart-home environment. We observed moderate to large correlations between predicted and ground-truth assessment scores, ranging from r = 0.601 to r = 0.871 for each clinical assessment.  more » « less
Award ID(s):
1954372
PAR ID:
10418201
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
ACM Transactions on Computing for Healthcare
Volume:
3
Issue:
4
ISSN:
2691-1957
Page Range / eLocation ID:
1 to 25
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; (, Frontiers in Digital Health)
    In recent years, behavioral markers such as spoken language and lexical preferences have been studied in the early detection of mild cognitive impairment (MCI) using conversations. While the combination of linguistic and acoustic signals have been shown to be effective in detecting MCI, they have generally been restricted to structured conversations in which the interviewee responds to fixed prompts. In this study, we show that linguistic and acoustic features can be combined synergistically to identify MCI in semi-structured conversations. Using conversational data from an on-going clinical trial ( Clinicaltrials.gov : NCT02871921), we find that the combination of linguistic and acoustic features on semi-structured conversations achieves a mean AUC of 82.7, significantly ( p < 0.01) out-performing linguistic-only (74.9 mean AUC) or acoustic-only (65.0 mean AUC) detections on hold-out data. Additionally, features (linguistic, acoustic and combination) obtained from semi-structured conversations outperform their counterparts obtained from structured weekly conversations in identifying MCI. Some linguistic categories are significantly better at predicting MCI status (e.g., death, home) than others. 
    more » « less
  2. ; ; ; ; ; ; ; ; ; ; et al (, Alzheimer's & Dementia)
    Abstract INTRODUCTIONAlzheimer's disease (AD) initiates years prior to symptoms, underscoring the importance of early detection. While amyloid accumulation starts early, individuals with substantial amyloid burden may remain cognitively normal, implying that amyloid alone is not sufficient for early risk assessment. METHODSGiven the genetic susceptibility of AD, a multi‐factorial pseudotime approach was proposed to integrate amyloid imaging and genotype data for estimating a risk score. Validation involved association with cognitive decline and survival analysis across risk‐stratified groups, focusing on patients with mild cognitive impairment (MCI). RESULTSOur risk score outperformed amyloid composite standardized uptake value ratio in correlation with cognitive scores. MCI subjects with lower pseudotime risk score showed substantial delayed onset of AD and slower cognitive decline. Moreover, pseudotime risk score demonstrated strong capability in risk stratification within traditionally defined subgroups such as early MCI, apolipoprotein E (APOE) ε4+ MCI,APOEε4– MCI, and amyloid+ MCI. DISCUSSIONOur risk score holds great potential to improve the precision of early risk assessment. HighlightsAccurate early risk assessment is critical for the success of clinical trials.A new risk score was built from integrating amyloid imaging and genetic data.Our risk score demonstrated improved capability in early risk stratification. 
    more » « less
  3. ; ; (, Journal of Alzheimer's Disease Reports)
    Background:Mild cognitive impairment (MCI) can be an early sign of Alzheimer’s disease and other types of dementia detectable through gait analysis. Curve walking, which demands greater cognitive and motor skills, may be more sensitive in MCI detection than straight walking. However, few studies have compared gait performance in older adults with and without MCI in these conditions. Objective:To compare the capability of curve and straight walking tests for the detection of MCI among older adults. Methods:We employed a Kinect v.2 camera to record the gait of 55 older adults (30 healthy controls, 25 with MCI) during single-task straight and curve walking tests. We examined 50 gait markers and conducted statistical analyses to compare groups and conditions. The trail was approved with protocol No. IR.SEMUMS.REC.1398.237 by the ethics committee of Semnan University of Medical Sciences in Iran. Results:Older adults with MCI exhibited more compromised gait performance, particularly during curve walking. Curve walking outperformed straight walking in MCI detection, with several gait markers showing significant differences between healthy controls and MCI patients. These markers encompass average velocity, cadence, temporal markers (e.g., gait cycle subphase durations), spatial markers (e.g., foot position changes during gait subphases), and spatiotemporal markers (e.g., step and stride velocities). Conclusions:Our study suggests curve walking as a more informative and challenging test for MCI detection among older adults, facilitating early diagnosis using non-invasive, cost-effective tools like the Kinect v.2 camera, complementing cognitive assessments in early diagnosis, and tracking MCI progression to dementia. 
    more » « less
  4. ; ; ; (, International Journal of Molecular Sciences)
    Alzheimer’s disease (AD) presents significant challenges in clinical practice due to its heterogeneous manifestation and variable progression rates. This work develops a comprehensive anatomical staging framework to predict progression from mild cognitive impairment (MCI) to AD. Using the ADNI database, the scalable Subtype and Stage Inference (s-SuStaIn) model was applied to 118 neuroanatomical features from cognitively normal (n = 504) and AD (n = 346) participants. The framework was validated on 808 MCI participants through associations with clinical progression, CSF and FDG-PET biomarkers, and neuropsychiatric measures, while adjusting for common confounders (age, gender, education, and APOE ε4 alleles). The framework demonstrated superior prognostic accuracy compared to traditional risk assessment (C-index = 0.73 vs. 0.62). Four distinct disease subtypes showed differential progression rates, biomarker profiles (FDG-PET and CSF Aβ42), and cognitive trajectories: Subtype 1, subcortical-first pattern; Subtype 2, executive–cortical pattern; Subtype 3, disconnection pattern; and Subtype 4, frontal–executive pattern. Stage-dependent changes revealed systematic deterioration across diverse cognitive domains, particularly in learning acquisition, visuospatial processing, and functional abilities. This data-driven approach captures clinically meaningful disease heterogeneity and improves prognostication in MCI, potentially enabling more personalized therapeutic strategies and clinical trial design. 
    more » « less
  5. ; ; (, Frontiers in Neurology)
    IntroductionAlzheimer's disease and related disorders (ADRD) progressively impair cognitive function, prompting the need for early detection to mitigate its impact. Mild Cognitive Impairment (MCI) may signal an early cognitive decline due to ADRD. Thus, developing an accessible, non-invasive method for detecting MCI is vital for initiating early interventions to prevent severe cognitive deterioration. MethodsThis study explores the utility of analyzing gait patterns, a fundamental aspect of human motor behavior, on straight and oval paths for diagnosing MCI. Using a Kinect v.2 camera, we recorded the movements of 25 body joints from 25 individuals with MCI and 30 healthy older adults (HC). Signal processing, descriptive statistical analysis, and machine learning techniques were employed to analyze the skeletal gait data in both walking conditions. Results and discussionThe study demonstrated that both straight and oval walking patterns provide valuable insights for MCI detection, with a notable increase in identifiable gait features in the more complex oval walking test. The Random Forest model excelled among various algorithms, achieving an 85.50% accuracy and an 83.9% F-score in detecting MCI during oval walking tests. This research introduces a cost-effective, Kinect-based method that integrates gait analysis—a key behavioral pattern—with machine learning, offering a practical tool for MCI screening in both clinical and home environments. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email