skip to main content


Title: Detecting Clinically Relevant Emotional Distress and Functional Impairment in Children and Adolescents: Protocol for an Automated Speech Analysis Algorithm Development Study
Background Even before the onset of the COVID-19 pandemic, children and adolescents were experiencing a mental health crisis, partly due to a lack of quality mental health services. The rate of suicide for Black youth has increased by 80%. By 2025, the health care system will be short of 225,000 therapists, further exacerbating the current crisis. Therefore, it is of utmost importance for providers, schools, youth mental health, and pediatric medical providers to integrate innovation in digital mental health to identify problems proactively and rapidly for effective collaboration with other health care providers. Such approaches can help identify robust, reproducible, and generalizable predictors and digital biomarkers of treatment response in psychiatry. Among the multitude of digital innovations to identify a biomarker for psychiatric diseases currently, as part of the macrolevel digital health transformation, speech stands out as an attractive candidate with features such as affordability, noninvasive, and nonintrusive. Objective The protocol aims to develop speech-emotion recognition algorithms leveraging artificial intelligence/machine learning, which can establish a link between trauma, stress, and voice types, including disrupting speech-based characteristics, and detect clinically relevant emotional distress and functional impairments in children and adolescents. Methods Informed by theoretical foundations (the Theory of Psychological Trauma Biomarkers and Archetypal Voice Categories), we developed our methodology to focus on 5 emotions: anger, happiness, fear, neutral, and sadness. Participants will be recruited from 2 local mental health centers that serve urban youths. Speech samples, along with responses to the Symptom and Functioning Severity Scale, Patient Health Questionnaire 9, and Adverse Childhood Experiences scales, will be collected using an Android mobile app. Our model development pipeline is informed by Gaussian mixture model (GMM), recurrent neural network, and long short-term memory. Results We tested our model with a public data set. The GMM with 128 clusters showed an evenly distributed accuracy across all 5 emotions. Using utterance-level features, GMM achieved an accuracy of 79.15% overall, while frame selection increased accuracy to 85.35%. This demonstrates that GMM is a robust model for emotion classification of all 5 emotions and that emotion frame selection enhances accuracy, which is significant for scientific evaluation. Recruitment and data collection for the study were initiated in August 2021 and are currently underway. The study results are likely to be available and published in 2024. Conclusions This study contributes to the literature as it addresses the need for speech-focused digital health tools to detect clinically relevant emotional distress and functional impairments in children and adolescents. The preliminary results show that our algorithm has the potential to improve outcomes. The findings will contribute to the broader digital health transformation. International Registered Report Identifier (IRRID) DERR1-10.2196/46970  more » « less
Award ID(s):
2126811
NSF-PAR ID:
10437974
Author(s) / Creator(s):
; ; ; ; ;
Date Published:
Journal Name:
JMIR Research Protocols
Volume:
12
ISSN:
1929-0748
Page Range / eLocation ID:
e46970
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    Psychological distress is the most common complication of pregnancy. High‐risk concerns can include severe emotion dysregulation, suicidality and self‐injury, and health risk behaviours, which bear substantial consequences for caregivers and families. Yet, effective, comprehensive interventions for high‐risk caregivers have received limited attention. Dialectical behaviour therapy (DBT) is a frontline treatment for such concerns. Accordingly, we conducted a scoping review on the implementation of DBT in the perinatal period. The Preferred Reporting Items for Systematic Reviews and Meta‐Analyses (PRISMA) guidelines were followed. Seven studies were identified; study designs included case studies and single‐arm pilot trials. Most studies used DBT‐informed protocols with significant adaptations, few included multiple components of DBT (i.e. skills group, individual therapy, phone coaching and consultation team), and none met criteria for adherent delivery of all four modes of DBT treatment. Findings suggest DBT‐informed interventions may be successfully implemented to treat a range of perinatal mental health symptoms, including borderline personality disorder, depression, anxiety, and post‐traumatic stress, and to promote emotion regulation and positive parenting behaviours. While results provide preliminary support for perinatal DBT, this literature is scant and empirical rigour considerably lacking. Clinical implications and future directions are outlined to aid researchers and providers in addressing the ongoing perinatal mental health crisis and developing sorely needed interventions to address the needs of high‐risk caregivers.

     
    more » « less
  2. In recent news, organizations have been considering the use of facial and emotion recognition for applications involving youth such as tackling surveillance and security in schools. However, the majority of efforts on facial emotion recognition research have focused on adults. Children, particularly in their early years, have been shown to express emotions quite differently than adults. Thus, before such algorithms are deployed in environments that impact the wellbeing and circumstance of youth, a careful examination should be made on their accuracy with respect to appropriateness for this target demographic. In this work, we utilize several datasets that contain facial expressions of children linked to their emotional state to evaluate eight different commercial emotion classification systems. We compare the ground truth labels provided by the respective datasets to the labels given with the highest confidence by the classification systems and assess the results in terms of matching score (TPR), positive predictive value, and failure to compute rate. Overall results show that the emotion recognition systems displayed subpar performance on the datasets of children's expressions compared to prior work with adult datasets and initial human ratings. We then identify limitations associated with automated recognition of emotions in children and provide suggestions on directions with enhancing recognition accuracy through data diversification, dataset accountability, and algorithmic regulation. 
    more » « less
  3. Boon-Peng, Hoh (Ed.)
    Responses to early life adversity differ greatly across individuals. Elucidating which factors underlie this variation can help us better understand how to improve health trajectories. Here we used a case:control study of refugee and non-refugee youth, differentially exposed to war-related trauma, to investigate the effects of genetics and psychosocial environment on response to trauma. We investigated genetic variants in two genes (serotonin transporter, 5-HTT , and catechol-O-methyltransferase, COMT ) that have been implicated in response to trauma. We collected buccal samples and survey data from 417 Syrian refugee and 306 Jordanian non-refugee youth who were enrolled in a randomized controlled trial to evaluate a mental health-focused intervention. Measures of lifetime trauma exposure, resilience, and six mental health and psychosocial stress outcomes were collected at three time points: baseline, ~13 weeks, and ~48 weeks. We used multilevel models to identify gene x environment (GxE) interactions and direct effects of the genetic variants in association with the six outcome measures over time. We did not identify any interactions with trauma exposure, but we did identify GxE interactions with both genes and resilience; 1) individuals with high expression (HE) variants of 5-HTTLPR and high levels of resilience had the lowest levels of perceived stress and 2) individuals homozygous for the Val variant of COMT with high levels of resilience showed stable levels of post-traumatic stress symptoms. We also identified a direct protective effect of 5-HTTLPR HE homozygotes on perceived insecurity. Our results point to novel interactions between the protective effects of genetic variants and resilience, lending support to ideas of differential susceptibility and altered stress reactivity in a cohort of war-affected adolescents. 
    more » « less
  4. Abstract

    Children make up over half of the world's migrants and refugees and face a multitude of traumatic experiences prior to, during, and following migration. Here, we focus on migrant children emigrating from Mexico and Central America to the United States and review trauma related to migration, as well as its implications for the mental health of migrant and refugee children. We then draw upon the early adversity literature to highlight potential behavioral and neurobiological sequalae of migration‐related trauma exposure, focusing on attachment, emotion regulation, and fear learning and extinction as transdiagnostic mechanisms underlying the development of internalizing and externalizing symptomatology following early‐life adversity. This review underscores the need for interdisciplinary efforts to both mitigate the effects of trauma faced by migrant and refugee youth emigrating from Mexico and Central America and, of primary importance, to prevent child exposure to trauma in the context of migration. Thus, we conclude by outlining policy recommendations aimed at improving the mental health of migrant and refugee youth.

     
    more » « less
  5. Theory—understanding mental processes that drive decisions—is important to help patients and providers make decisions that reflect medical advances and personal values. Building on a 2008 review, we summarize current tenets of fuzzy-trace theory (FTT) in light of new evidence that provides insight regarding mental representations of options and how such representations connect to values and evoke emotions. We discuss implications for communicating risks, preventing risky behaviors, discouraging misinformation, and choosing appropriate treatments. Findings suggest that simple, fuzzy but meaningful gist representations of information often determine decisions. Within minutes of conversing with their doctor, reading a health-related web post, or processing other health information, patients rely on gist memories of that information rather than verbatim details. This fuzzy-processing preference explains puzzles and paradoxes in how patients (and sometimes providers) think about probabilities (e.g., “50-50” chance), outcomes of treatment (e.g., with antibiotics), experiences of pain, end-of-life decisions, memories for medication instructions, symptoms of concussion, and transmission of viruses (e.g., in AIDS and COVID-19). As examples, participation in clinical trials or seeking treatments with low probabilities of success (e.g., with antibiotics or at the end of life) may indicate a defensibly different categorical gist perspective on risk as opposed to simply misunderstanding probabilities or failing to make prescribed tradeoffs. Thus, FTT explains why people avoid precise tradeoffs despite computing them. Facilitating gist representations of information offers an alternative approach that goes beyond providing uninterpreted “neutral” facts versus persuading or shifting the balance between fast versus slow thinking (or emotion vs. cognition). In contrast to either taking mental shortcuts or deliberating about details, gist processing facilitates application of advanced knowledge and deeply held values to choices. Highlights Fuzzy-trace theory (FTT) supports practical approaches to improving health and medicine. FTT differs in important respects from other theories of decision making, which has implications for how to help patients, providers, and health communicators. Gist mental representations emphasize categorical distinctions, reflect understanding in context, and help cue values relevant to health and patient care. Understanding the science behind theory is crucial for evidence-based medicine. 
    more » « less