An official website of the United States government
Emotions provide critical information regarding a person's health and well-being. Therefore, the ability to track emotion and patterns in emotion over time could provide new opportunities in measuring health longitudinally. This is of particular importance for individuals with bipolar disorder (BD), where emotion dysregulation is a hallmark symptom of increasing mood severity. However, measuring emotions typically requires self-assessment, a willful action outside of one's daily routine. In this paper, we describe a novel approach for collecting real-world natural speech data from daily life and measuring emotions from these data. The approach combines a novel data collection pipeline and validated robust emotion recognition models. We describe a deployment of this pipeline that included parallel clinical and self-report measures of mood and self-reported measures of emotion. Finally, we present approaches to estimate clinical and self-reported mood measures using a combination of passive and self-reported emotion measures. The results demonstrate that both passive and self-reported measures of emotion contribute to our ability to accurately estimate mood symptom severity for individuals with BD.
more »
« less
Identifying Mood Episodes Using Dialogue Features from Clinical Interviews
Bipolar disorder, a severe chronic mental illness characterized by pathological mood swings from depression to mania, requires ongoing symptom severity tracking to both guide and measure treatments that are critical for maintaining long-term health. Mental health professionals assess symptom severity through semi-structured clinical interviews. During these interviews, they observe their patients’ spoken behaviors, including both what the patients say and how they say it. In this work, we move beyond acoustic and lexical information, investigating how higher-level interactive patterns also change during mood episodes. We then perform a secondary analysis, asking if these interactive patterns, measured through dialogue features, can be used in conjunction with acoustic features to automatically recognize mood episodes. Our results show that it is beneficial to consider dialogue features when analyzing and building automated systems for predicting and monitoring mood.
more »
« less
- Award ID(s):
- 1651740
- PAR ID:
- 10125072
- Date Published:
- Journal Name:
- Interspeech
- Page Range / eLocation ID:
- 1926 to 1930
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Background Supporting mental health and wellness is of increasing interest due to a growing recognition of the prevalence and burden of mental health issues. Mood is a central aspect of mental health, and several technologies, especially mobile apps, have helped people track and understand it. However, despite formative work on and dissemination of mood-tracking apps, it is not well understood how mood-tracking apps used in real-world contexts might benefit people and what people hope to gain from them. Objective To address this gap, the purpose of this study was to understand motivations for and experiences in using mood-tracking apps from people who used them in real-world contexts. Methods We interviewed 22 participants who had used mood-tracking apps using a semistructured interview and card sorting task. The interview focused on their experiences using a mood-tracking app. We then conducted a card sorting task using screenshots of various data entry and data review features from mood-tracking apps. We used thematic analysis to identify themes around why people use mood-tracking apps, what they found useful about them, and where people felt these apps fell short. Results Users of mood-tracking apps were primarily motivated by negative life events or shifts in their own mental health that prompted them to engage in tracking and improve their situation. In general, participants felt that using a mood-tracking app facilitated self-awareness and helped them to look back on a previous emotion or mood experience to understand what was happening. Interestingly, some users reported less inclination to document their negative mood states and preferred to document their positive moods. There was a range of preferences for personalization and simplicity of tracking. Overall, users also liked features in which their previous tracked emotions and moods were visualized in figures or calendar form to understand trends. One gap in available mood-tracking apps was the lack of app-facilitated recommendations or suggestions for how to interpret their own data or improve their mood. Conclusions Although people find various features of mood-tracking apps helpful, the way people use mood-tracking apps, such as avoiding entering negative moods, tracking infrequently, or wanting support to understand or change their moods, demonstrate opportunities for improvement. Understanding why and how people are using current technologies can provide insights to guide future designs and implementations.more » « less
-
Psychotic disorders are forms of severe mental illness characterized by abnormal social function and a general sense of disconnect with reality. The evaluation of such disorders is often complex, as their multifaceted nature is often difficult to quantify. Multimodal behavior analysis technologies have the potential to help address this need and supply timelier and more objective decision support tools in clinical settings. While written language and nonverbal behaviors have been previously studied, the present analysis takes the novel approach of examining the rarely-studied modality of spoken language of individuals with psychosis as naturally used in social, face-to-face interactions. Our analyses expose a series of language markers associated with psychotic symptom severity, as well as interesting interactions between them. In particular, we examine three facets of spoken language: (1) lexical markers, through a study of the function of words; (2) structural markers, through a study of grammatical fluency; and (3) disfluency markers, through a study of dialogue self-repair. Additionally, we develop predictive models of psychotic symptom severity, which achieve significant predictive power on both positive and negative psychotic symptom scales. These results constitute a significant step toward the design of future multimodal clinical decision support tools for computational phenotyping of mental illness.more » « less
-
Bipolar Disorder, a mood disorder with recurrent mania and depression, requires ongoing monitoring and specialty management. Current monitoring strategies are clinically-based, engaging highly specialized medical professionals who are becoming increasingly scarce. Automatic speech-based monitoring via smartphones has the potential to augment clinical monitoring by providing inexpensive and unobtrusive measurements of a patient’s daily life. The success of such an approach is contingent on the ability to successfully utilize “in-the-wild” data. However, most existing work on automatic mood detection uses datasets collected in clinical or laboratory settings. This study presents experiments in automatically detecting depression severity in individuals with Bipolar Disorder using data derived from clinical interviews and from personal conversations. We find that mood assessment is more accurate using data collected from clinical interactions, in part because of their highly structured nature. We demonstrate that although the features that are most effective in clinical interactions do not extend well to personal conversational data, we can identify alternative features relevant in personal conversational speech to detect mood symptom severity. Our results highlight the challenges unique to working with “in-the-wild” data, providing insight into the degree to which the predictive ability of speech features is preserved outside of a clinical interview.more » « less
-
In this mini-review, we discuss the fundamentals of using technology in mental health diagnosis and tracking. We highlight those principles using two clinical concepts: (1) cravings and relapse in the context of addictive disorders and (2) anhedonia in the context of depression. This manuscript is useful for both clinicians wanting to understand the scope of technology use in psychiatry and for computer scientists and engineers wishing to assess psychiatric frameworks useful for diagnosis and treatment. The increase in smartphone ownership and internet connectivity, as well as the accelerated development of wearable devices, have made the observation and analysis of human behavior patterns possible. This has, in turn, paved the way to understand mental health conditions better. These technologies have immense potential in facilitating the diagnosis and tracking of mental health conditions; they also allow the implementation of existing behavioral treatments in new contexts (e.g., remotely, online, and in rural/underserved areas), and the possibility to develop new treatments based on new understanding of behavior patterns. The path to understand how to best use technology in mental health includes the need to match interdisciplinary frameworks from engineering/computer sciences and psychiatry. Thus, we start our review by introducing bio-behavioral sensing, the types of information available, and what behavioral patterns they may reflect and be related to in psychiatric diagnostic frameworks. This information is linked to the use of functional imaging, highlighting how imaging modalities can be considered “ground truth” for mental health/psychiatric dimensions, given the heterogeneity of clinical presentations, and the difficulty of determining what symptom corresponds to what disease. We then discuss how mental health/psychiatric dimensions overlap, yet differ from, psychiatric diagnoses. Using two clinical examples, we highlight the potential agreement areas in assessment/management of anhedonia and cravings. These two dimensions were chosen because of their link to two very prevalent diseases worldwide: depression and addiction. Anhedonia is a core symptom of depression, which is one of the leading causes of disability worldwide. Cravings, the urge to use a substance or perform an action (e.g., shopping, internet), is the leading step before relapse. Lastly, through the manuscript, we discuss potential mental health dimensions.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.21437/interspeech.2019-1878
