skip to main content


Title: Automated measurement of head movement synchrony during dyadic depression severity interviews
With few exceptions, most research in automated assessment of depression has considered only the patient’s behavior to the exclusion of the therapist’s behavior. We investigated the interpersonal coordination (synchrony) of head movement during patient-therapist clinical interviews. Participants with major depressive disorder were recorded in clinical interviews (Hamilton Rating Scale for Depression, HRSD) at 7-week intervals over a period of 21 weeks. For each session, patient and therapist 3D head movement was tracked from 2D videos. Head angles in the horizontal (pitch) and vertical (yaw) axes were used to measure head movement. Interpersonal coordination of head movement between patients and therapists was measured using windowed cross-correlation. Patterns of coordination in head movement were investigated using the peak picking algorithm. Changes in head movement coordination over the course of treatment were measured using a hierarchical linear model (HLM). The results indicated a strong effect for patient-therapist head movement synchrony. Within-dyad variability in head movement coordination was higher than between-dyad variability, meaning that differences over time in a dyad were higher as compared to the differences between dyads. Head movement synchrony did not change over the course of treatment. To the best of our knowledge, this study is the first attempt to analyze the mutual influence of patient-therapist head movement in relation to depression severity.  more » « less
Award ID(s):
1721667
NSF-PAR ID:
10100353
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition
Page Range / eLocation ID:
1-8
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Background & Purpose: Deformational plagiocephaly and brachycephaly (DPB) is a cranial condition manifested in 20% of infants in the US. DPB affects children and their families through psychological pressure, social stigma, and significant medical costs. If detected between 0-3 months of age, there is strong potential for correction via aggressive repositioning and/or physical therapy if congenital muscular torticollis is present. At later stages, DPB is most effectively treated by more expensive treatments like helmet therapy. Two cranial parameters that can help with the early detection and tracking of DPB are the cranial index (CI) and cranial vault asymmetry index (CVAI). Currently, these measurements are performed with a hand caliper by a specialist, i.e., nurse practitioner (CRNP) or physician assistant who specializes in cleft-craniofacial diagnosis, physical therapist, pediatric plastic/neurosurgeons, or orthotist. To make the measurements frequent, accessible, and accurate at the point of care, i.e., in pediatric offices, we developed and evaluated a mobile app called SoftspotTM to measure CI and CVAI, thus facilitating the early detection and monitoring of DPB. Method/Description: Sequences of bird’s eye-view head photos extracted from video were collected for 77 patients (aged 2 – 11 months, 51 females, 26 males) with an iPhone X (Apple Inc., Cupertino, CA). The head length, width, and diagonals were measured by a single CRNP via hand calipers at a large multidisciplinary cranio-facial center with IRB approval and patient consent. For each patient, five images were chosen by an analyst and segmented into head components, namely the head and nose, using quantitative imaging methods. For each image CI and CVAI were automatically measured, and these measurements were averaged for each patient. Automated CI and CVAI measurements were compared to values obtained by the caliper measurements in terms of mean absolute error (MAE), and outliers were excluded beyond 3 standard deviations away from the average MAE. Results were further analyzed by the Bland-Altman method and Spearman Correlation Coefficient. Results: MAE was 2.18 ± 1.60 for CI and 1.57 ± 1.03 for CVAI measurements. Spearman Correlation Coefficients between measurements and ground truth were 0.93 for CI (p<0.001) and 0.91 for CVAI (p<0.001). Bland-Altman analysis revealed limits of agreement for CI and CVAI as [-4.59, 5.76] (mean = 0.59) and [-3.91, 3.40] (mean = -0.25) respectively. Conclusions: Digital smartphone-based methods for DPB assessment are feasible, and this study demonstrated significant correlation between automated digital measurements and ground truth clinical values. Smartphone-based measurements of DPB can be performed at the point of care to improve the early detection and treatment of DPB. 
    more » « less
  2. Observing how infants and mothers coordinate their behaviors can highlight meaningful patterns in early communication and infant development. While dyads often differ in the modalities they use to communicate, especially in the first year of life, it remains unclear how to capture coordination across multiple types of behaviors using existing computational models of interpersonal synchrony. This paper explores Dynamic Mode Decomposition with control (DMDc) as a method of integrating multiple signals from each communicating partner into a model of multimodal behavioral coordination. We used an existing video dataset to track the head pose, arm pose, and vocal fundamental frequency of infants and mothers during the Face-to-Face Still-Face (FFSF) procedure, a validated 3-stage interaction paradigm. For each recorded interaction, we fit both unimodal and multimodal DMDc models to the extracted pose data. The resulting dynamic characteristics of the models were analyzed to evaluate trends in individual behaviors and dyadic processes across infant age and stages of the interactions. Results demonstrate that observed trends in interaction dynamics across stages of the FFSF protocol were stronger and more significant when models incorporated both head and arm pose data, rather than a single behavior modality. Model output showed significant trends across age, identifying changes in infant movement and in the relationship between infant and mother behaviors. Models that included mothers’ audio data demonstrated similar results to those evaluated with pose data, confirming that DMDc can leverage different sets of behavioral signals from each interacting partner. Taken together, our results demonstrate the potential of DMDc toward integrating multiple behavioral signals into the measurement of multimodal interpersonal coordination. 
    more » « less
  3. This study investigates the presence of dynamical patterns of interpersonal coordination in extended deceptive conversations across multimodal channels of behavior. Using a novel "devil’s advocate" paradigm, we experimentally elicited deception and truth across topics in which conversational partners either agreed or disagreed, and where one partner was surreptitiously asked to argue an opinion opposite of what he or she really believed. We focus on interpersonal coordination as an emergent behavioral signal that captures interdependencies between conversational partners, both as the coupling of head movements over the span of milliseconds, measured via a windowed lagged cross correlation (WLCC) technique, and more global temporal dependencies across speech rate, using cross recurrence quantification analysis (CRQA). Moreover, we considered how interpersonal coordination might be shaped by strategic, adaptive conversational goals associated with deception. We found that deceptive conversations displayed more structured speech rate and higher head movement coordination, the latter with a peak in deceptive disagreement conversations. Together the results allow us to posit an adaptive account, whereby interpersonal coordination is not beholden to any single functional explanation, but can strategically adapt to diverse conversational demands. 
    more » « less
  4. Abstract Background

    In order to accurately accumulate delivered dose for head and neck cancer patients treated with the Adapt to Position workflow on the 1.5T magnetic resonance imaging (MRI)‐linear accelerator (MR‐linac), the low‐resolution T2‐weighted MRIs used for daily setup must be segmented to enable reconstruction of the delivered dose at each fraction.

    Purpose

    In this pilot study, we evaluate various autosegmentation methods for head and neck organs at risk (OARs) on on‐board setup MRIs from the MR‐linac for off‐line reconstruction of delivered dose.

    Methods

    Seven OARs (parotid glands, submandibular glands, mandible, spinal cord, and brainstem) were contoured on 43 images by seven observers each. Ground truth contours were generated using a simultaneous truth and performance level estimation (STAPLE) algorithm. Twenty total autosegmentation methods were evaluated in ADMIRE: 1–9) atlas‐based autosegmentation using a population atlas library (PAL) of 5/10/15 patients with STAPLE, patch fusion (PF), random forest (RF) for label fusion; 10–19) autosegmentation using images from a patient's 1–4 prior fractions (individualized patient prior [IPP]) using STAPLE/PF/RF; 20) deep learning (DL) (3D ResUNet trained on 43 ground truth structure sets plus 45 contoured by one observer). Execution time was measured for each method. Autosegmented structures were compared to ground truth structures using the Dice similarity coefficient, mean surface distance (MSD), Hausdorff distance (HD), and Jaccard index (JI). For each metric and OAR, performance was compared to the inter‐observer variability using Dunn's test with control. Methods were compared pairwise using the Steel‐Dwass test for each metric pooled across all OARs. Further dosimetric analysis was performed on three high‐performing autosegmentation methods (DL, IPP with RF and 4 fractions [IPP_RF_4], IPP with 1 fraction [IPP_1]), and one low‐performing (PAL with STAPLE and 5 atlases [PAL_ST_5]). For five patients, delivered doses from clinical plans were recalculated on setup images with ground truth and autosegmented structure sets. Differences in maximum and mean dose to each structure between the ground truth and autosegmented structures were calculated and correlated with geometric metrics.

    Results

    DL and IPP methods performed best overall, all significantly outperforming inter‐observer variability and with no significant difference between methods in pairwise comparison. PAL methods performed worst overall; most were not significantly different from the inter‐observer variability or from each other. DL was the fastest method (33 s per case) and PAL methods the slowest (3.7–13.8 min per case). Execution time increased with a number of prior fractions/atlases for IPP and PAL. For DL, IPP_1, and IPP_RF_4, the majority (95%) of dose differences were within ± 250 cGy from ground truth, but outlier differences up to 785 cGy occurred. Dose differences were much higher for PAL_ST_5, with outlier differences up to 1920 cGy. Dose differences showed weak but significant correlations with all geometric metrics (R2 between 0.030 and 0.314).

    Conclusions

    The autosegmentation methods offering the best combination of performance and execution time are DL and IPP_1. Dose reconstruction on on‐board T2‐weighted MRIs is feasible with autosegmented structures with minimal dosimetric variation from ground truth, but contours should be visually inspected prior to dose reconstruction in an end‐to‐end dose accumulation workflow.

     
    more » « less
  5. Background

    In recent years, epidemiological and clinical studies have revealed that depressive disorders can present in early childhood. To clarify the validity and prognostic significance of early childhood‐onset depression, we investigated diagnostic and functional outcomes in later childhood and adolescence.

    Methods

    A community sample (N = 516) was assessed for psychopathology at ages 3 and 6 using the Preschool Age Psychiatric Assessment. When participants were 9, 12, and 15 years old, children and parents completed the Kiddie Schedule for Affective Disorders and Schizophrenia and measures of symptoms and functioning.

    Results

    In models adjusting for covariates, depressed 3/6‐year‐old children were more likely to experience subsequent episodes of depressive disorders and exhibited significantly higher rates of later anxiety disorder, attention deficit hyperactivity disorder, and suicidality compared to children without depressive disorders at age 3/6. Early childhood depression was also associated with higher levels of mother, but not child, reported depressive symptoms at age 15 compared to children without depressive disorders at age 3/6. Finally, depression at age 3/6 predicted lower levels of global and interpersonal functioning and higher rates of treatment at age 15 compared to children without depressive disorders at age 3/6.

    Conclusions

    Results support the clinical significance of depression in 3/6‐year‐old children, although further studies with larger samples are needed.

     
    more » « less