An official website of the United States government
Artificial Intelligence (AI) systems for mental healthcare (MHCare) have been ever-growing after realizing the importance of early interventions for patients with chronic mental health (MH) conditions. Social media (SocMedia) emerged as the go-to platform for supporting patients seeking MHCare. The creation of peer-support groups without social stigma has resulted in patients transitioning from clinical settings to SocMedia supported interactions for quick help. Researchers started exploring SocMedia content in search of cues that showcase correlation or causation between different MH conditions to design better interventional strategies. User-level Classification-based AI systems were designed to leverage diverse SocMedia data from various MH conditions, to predict MH conditions. Subsequently, researchers created classification schemes to measure the severity of each MH condition. Such ad-hoc schemes, engineered features, and models not only require a large amount of data but fail to allow clinically acceptable and explainable reasoning over the outcomes. To improve Neural-AI for MHCare, infusion of clinical symbolic knowledge that clinicans use in decision making is required. An impactful use case of Neural-AI systems in MH is conversational systems. These systems require coordination between classification and generation to facilitate humanistic conversation in conversational agents (CA). Current CAs with deep language models lack factual correctness, medical relevance, and safety in their generations, which intertwine with unexplainable statistical classification techniques. This lecture-style tutorial will demonstrate our investigations into Neuro-symbolic methods of infusing clinical knowledge to improve the outcomes of Neural-AI systems to improve interventions for MHCare:(a) We will discuss the use of diverse clinical knowledge in creating specialized datasets to train Neural-AI systems effectively. (b) Patients with cardiovascular disease express MH symptoms differently based on gender differences. We will show that knowledge-infused Neural-AI systems can identify gender-specific MH symptoms in such patients. (c) We will describe strategies for infusing clinical process knowledge as heuristics and constraints to improve language models in generating relevant questions and responses.
more »
« less
Learning to Automate Follow-up Question Generation using Process Knowledge for Depression Triage on Reddit Posts
Conversational Agents (CAs) powered with deep language models (DLMs) have shown tremendous promise in the domain of mental health. Prominently, the CAs have been used to provide informational or therapeutic services (e.g., cognitive behavioral therapy) to patients. However, the utility of CAs to assist in mental health triaging has not been explored in the existing work as it requires a controlled generation of follow-up questions (FQs), which are often initiated and guided by the mental health professionals (MHPs) in clinical settings. In the context of ‘depression’, our experiments show that DLMs coupled with process knowledge in a mental health questionnaire generate 12.54% and 9.37% better FQs based on similarity and longest common subsequence matches to questions in the PHQ-9 dataset respectively, when compared with DLMs without process knowledge support.Despite coupling with process knowledge, we find that DLMs are still prone to hallucination, i.e., generating redundant, irrelevant, and unsafe FQs. We demonstrate the challenge of using existing datasets to train a DLM for generating FQs that adhere to clinical process knowledge. To address this limitation, we prepared an extended PHQ-9 based dataset, PRIMATE, in collaboration with MHPs. PRIMATE contains annotations regarding whether a particular question in the PHQ-9 dataset has already been answered in the user’s initial description of the mental health condition. We used PRIMATE to train a DLM in a supervised setting to identify which of the PHQ-9 questions can be answered directly from the user’s post and which ones would require more information from the user. Using performance analysis based on MCC scores, we show that PRIMATE is appropriate for identifying questions in PHQ-9 that could guide generative DLMs towards controlled FQ generation (with minimal hallucination) suitable for aiding triaging. The dataset created as a part of this research can be obtained from https://github.com/primate-mh/Primate2022
more »
« less
- Award ID(s):
- 2133842
- PAR ID:
- 10429256
- Date Published:
- Journal Name:
- Proceedings of the Eighth Workshop on Computational Linguistics and Clinical Psychology
- Page Range / eLocation ID:
- 137 to 147
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)The development of tools that promote active learning in engineering disciplines is critical. It is widely understood that students engaged in active learning environments outperform those taught using passive methods. Previously, we reported on the development and implementation of hands-on Low-Cost Desktop Learning Modules (LCDLMs) that replicate real-world industrial equipment which serves to create active learning environments. Thus far, miniaturized venturi meter, hydraulic loss, and double-pipe and shell & tube heat exchanger DLMs have been utilized by hundreds of students across the country. It was demonstrated that the use of DLMs in face-to-face classrooms results in statistically significant improvements in student performance as well as increases in student motivation compared to students taught in a traditional lecture-only style classroom. Last year, participants in the project conducted 45 implementations including over 600 DLMs at 24 universities across the country reaching more than 1,000 students. In this project, we report on the significant progress made in broad dissemination of DLMs and accompanying pedagogy. We demonstrate that DLMs serve to increase student learning gains not only in face-toface environments but also in virtual learning environments. Instructional videos were developed to aid in DLM-based learning during the COVID-19 pandemic when instructors were limited to virtual instruction. Preliminary results from this work show that students working with DLMs even in a virtual setting significantly outperform those taught without DLM-associated materials. Significant progress has also been made on the development of a new DLM cartridge: a see-through 3Dprinted miniature fluidized bed. The new 3D printing methodology will allow for rapid prototyping and streamlined development of DLMs. A 3D-printed evaporative cooling tower DLM will also be developed in the coming year. In October 2020, the team held a virtual implementers workshop to train new participating faculty in DLM use and implementation. In total, 13 new faculty participants from 10 universities attended the 6-hour, 2- day workshop and plan to implement DLMs in their classrooms during this academic year. In the last year, this project was disseminated in 8 presentations at the ASEE Virtual Conference (June 2020) and American Institute of Chemical Engineers Annual Conference (November 2019) as well as the AIChE virtual Community of Practice Labs Group and a seminar at a major university, ultimately disseminating DLM pedagogy to approximately 200 individuals including approximately 120 university faculty. Further, the former group postdoc has accepted an instructor faculty position at University of Wisconsin Madison where she will teach unit operations among other subjects; she and the remainder of the team believe the LCDLM project has prepared her well for that position. In the remaining 2.5 years of the project, we will continue to evaluate the effectiveness of DLMs in teaching key heat transfer and fluid dynamics concepts thru implementations in the rapidly expanding pool of participating universities. Further, we continue our ongoing efforts in creating the robust support structure necessary for large-scale adoption of hands-on educational tools for promotion of hands-on interactive student learning.more » « less
-
null (Ed.)The development of tools that promote active learning in engineering disciplines is critical. It is widely understood that students engaged in active learning environments outperform those taught using passive methods. Previously, we reported on the development and implementation of hands-on Low-Cost Desktop Learning Modules (LCDLMs) that replicate real-world industrial equipment which serves to create active learning environments. Thus far, miniaturized venturi meter, hydraulic loss, and double-pipe and shell & tube heat exchanger DLMs have been utilized by hundreds of students across the country. It was demonstrated that the use of DLMs in face-to-face classrooms results in statistically significant improvements in student performance as well as increases in student motivation compared to students taught in a traditional lecture-only style classroom. Last year, participants in the project conducted 45 implementations including over 600 DLMs at 24 universities across the country reaching more than 1,000 students. In this project, we report on the significant progress made in broad dissemination of DLMs and accompanying pedagogy. We demonstrate that DLMs serve to increase student learning gains not only in face-to-face environments but also in virtual learning environments. Instructional videos were developed to aid in DLM-based learning during the COVID-19 pandemic when instructors were limited to virtual instruction. Preliminary results from this work show that students working with DLMs even in a virtual setting significantly outperform those taught without DLM-associated materials. Significant progress has also been made on the development of a new DLM cartridge: a see-through 3D-printed miniature fluidized bed. The new 3D printing methodology will allow for rapid prototyping and streamlined development of DLMs. A 3D-printed evaporative cooling tower DLM will also be developed in the coming year. In October 2020, the team held a virtual implementers workshop to train new participating faculty in DLM use and implementation. In total, 13 new faculty participants from 10 universities attended the 6-hour, 2-day workshop and plan to implement DLMs in their classrooms during this academic year. In the last year, this project was disseminated in 8 presentations at the American Society for Engineering Education (ASEE) Virtual Conference (June 2020) and American Institute of Chemical Engineers Annual Conference (November 2019) as well as the AIChE virtual Community of Practice Labs Group and a seminar at a major university, ultimately disseminating DLM pedagogy to approximately 200 individuals including approximately 120 university faculty. Further, the former group postdoc has accepted an instructor faculty position at University of Wisconsin Madison where she will teach unit operations among other subjects; she and the remainder of the team believe the LCDLM project has prepared her well for that position. In the remaining 2.5 years of the project, we will continue to evaluate the effectiveness of DLMs in teaching key heat transfer and fluid dynamics concepts thru implementations in the rapidly expanding pool of participating universities. Further, we continue our ongoing efforts in creating the robust support structure necessary for large-scale adoption of hands-on educational tools for promotion of hands-on interactive student learning.more » « less
-
The aim of this study was to assess the correlation of depression and anxiety with time spent at home among students at two universities—one urban and the other suburban—during the COVID-19 pandemic. Methods: Geolocation data from the smartphones of 124 participants were collected between February 2021 and May 2021. The level of depression was estimated by the PHQ-9 and PHQ-2 screening tools, and anxiety scores were estimated by the GAD-2 and GAD-7 screening tools. Results: 51% of participants in the PHQ-9 surveys indicated mild to severe depression. Participants spent on average 75% of their time at home during COVID. Time spent at home had a positive correlation with the mental health of urban students but a negative correlation with suburban students. The relation between the time at home with mental health was stronger among female participants than among male participants. Correlations between female depression, anxiety, and time at home were significant. Conclusions: Lockdown and distance learning contributed to the high levels of depression in university students. This research highlights the importance of time spent at home for mental health being during the pandemic and the importance of distinguishing between urban and suburban settings when formulating public health recommendations. Quality of time spent at home versus time spent outside differentiated the mental well-being of students located in different environments. Staying at home may be recommended for students without access to safe outdoor places as it is associated with lower levels of depression.more » « less
-
Language models have the potential to assess mental health using social media data. By analyzing online posts and conversations, these models can detect patterns indicating mental health conditions like depression, anxiety, or suicidal thoughts. They examine keywords, language markers, and sentiment to gain insights into an individual’s mental well-being. This information is crucial for early detection, intervention, and support, improving mental health care and prevention strategies. However, using language models for mental health assessments from social media has two limitations: (1) They do not compare posts against clinicians’ diagnostic processes, and (2) It’s challenging to explain language model outputs using concepts that the clinician can understand, i.e., clinician-friendly explanations. In this study, we introduce Process Knowledge-infused Learning (PK-iL), a new learning paradigm that layers clinical process knowledge structures on language model outputs, enabling clinician-friendly explanations of the underlying language model predictions. We rigorously test our methods on existing benchmark datasets, augmented with such clinical process knowledge, and release a new dataset for assessing suicidality. PKiL performs competitively, achieving a 70% agreement with users, while other XAI methods only achieve 47% agreement (average inter-rater agreement of 0.72). Our evaluations demonstrate that PK-iL effectively explains model predictions to clinicians.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.18653/v1/2022.clpsych-1.12
