An official website of the United States government
IntroductionWith the increasing utilization of text-based suicide crisis counseling, new means of identifying at risk clients must be explored. Natural language processing (NLP) holds promise for evaluating the content of crisis counseling; here we use a data-driven approach to evaluate NLP methods in identifying client suicide risk. MethodsDe-identified crisis counseling data from a regional text-based crisis encounter and mobile tipline application were used to evaluate two modeling approaches in classifying client suicide risk levels. A manual evaluation of model errors and system behavior was conducted. ResultsThe neural model outperformed a term frequency-inverse document frequency (tf-idf) model in the false-negative rate. While 75% of the neural model’s false negative encounters had some discussion of suicidality, 62.5% saw a resolution of the client’s initial concerns. Similarly, the neural model detected signals of suicidality in 60.6% of false-positive encounters. DiscussionThe neural model demonstrated greater sensitivity in the detection of client suicide risk. A manual assessment of errors and model performance reflected these same findings, detecting higher levels of risk in many of the false-positive encounters and lower levels of risk in many of the false negatives. NLP-based models can detect the suicide risk of text-based crisis encounters from the encounter’s content.
more »
« less
An NLP approach to identify SDoH-related circumstance and suicide crisis from death investigation narratives
Abstract ObjectivesSuicide presents a major public health challenge worldwide, affecting people across the lifespan. While previous studies revealed strong associations between Social Determinants of Health (SDoH) and suicide deaths, existing evidence is limited by the reliance on structured data. To resolve this, we aim to adapt a suicide-specific SDoH ontology (Suicide-SDoHO) and use natural language processing (NLP) to effectively identify individual-level SDoH-related social risks from death investigation narratives. Materials and MethodsWe used the latest National Violent Death Report System (NVDRS), which contains 267 804 victim suicide data from 2003 to 2019. After adapting the Suicide-SDoHO, we developed a transformer-based model to identify SDoH-related circumstances and crises in death investigation narratives. We applied our model retrospectively to annotate narratives whose crisis variables were not coded in NVDRS. The crisis rates were calculated as the percentage of the group’s total suicide population with the crisis present. ResultsThe Suicide-SDoHO contains 57 fine-grained circumstances in a hierarchical structure. Our classifier achieves AUCs of 0.966 and 0.942 for classifying circumstances and crises, respectively. Through the crisis trend analysis, we observed that not everyone is equally affected by SDoH-related social risks. For the economic stability crisis, our result showed a significant increase in crisis rate in 2007–2009, parallel with the Great Recession. ConclusionsThis is the first study curating a Suicide-SDoHO using death investigation narratives. We showcased that our model can effectively classify SDoH-related social risks through NLP approaches. We hope our study will facilitate the understanding of suicide crises and inform effective prevention strategies.
more »
« less
- Award ID(s):
- 2019844
- PAR ID:
- 10433260
- Publisher / Repository:
- Oxford University Press
- Date Published:
- Journal Name:
- Journal of the American Medical Informatics Association
- Volume:
- 30
- Issue:
- 8
- ISSN:
- 1067-5027
- Format(s):
- Medium: X Size: p. 1408-1417
- Size(s):
- p. 1408-1417
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract ObjectiveThe impact of social determinants of health (SDoH) on patients’ healthcare quality and the disparity is well known. Many SDoH items are not coded in structured forms in electronic health records. These items are often captured in free-text clinical notes, but there are limited methods for automatically extracting them. We explore a multi-stage pipeline involving named entity recognition (NER), relation classification (RC), and text classification methods to automatically extract SDoH information from clinical notes. Materials and MethodsThe study uses the N2C2 Shared Task data, which were collected from 2 sources of clinical notes: MIMIC-III and University of Washington Harborview Medical Centers. It contains 4480 social history sections with full annotation for 12 SDoHs. In order to handle the issue of overlapping entities, we developed a novel marker-based NER model. We used it in a multi-stage pipeline to extract SDoH information from clinical notes. ResultsOur marker-based system outperformed the state-of-the-art span-based models at handling overlapping entities based on the overall Micro-F1 score performance. It also achieved state-of-the-art performance compared with the shared task methods. Our approach achieved an F1 of 0.9101, 0.8053, and 0.9025 for Subtasks A, B, and C, respectively. ConclusionsThe major finding of this study is that the multi-stage pipeline effectively extracts SDoH information from clinical notes. This approach can improve the understanding and tracking of SDoHs in clinical settings. However, error propagation may be an issue and further research is needed to improve the extraction of entities with complex semantic meanings and low-frequency entities. We have made the source code available at https://github.com/Zephyr1022/SDOH-N2C2-UTSA.more » « less
-
Social factors like family background, education level, financial status, and stress can impact public health outcomes, such as suicidal ideation. However, the analysis of social factors for suicide prevention has been limited by the lack of up-to-date suicide reporting data, variations in reporting practices, and small sample sizes. In this study, we analyzed 172,629 suicide incidents from 2014 to 2020 utilizing the National Violent Death Reporting System Restricted Access Database (NVDRS-RAD). Logistic regression models were developed to examine the relationships between demographics and suicide-related circumstances. Trends over time were assessed, and Latent Dirichlet Allocation (LDA) was used to identify common suiciderelated social factors. Mental health, interpersonal relationships, mental health treatment and disclosure, and school/work-related stressors were identified as the main themes of suicide-related social factors. This study also identified systemic disparities across various population groups, particularly concerning Black individuals, young people aged under 24, healthcare practitioners, and those with limited education backgrounds, which shed light on potential directions for demographic-specific suicidal interventions.more » « less
-
Abstract BackgroundPrevious social determinants of health (SDoH) studies on laryngeal cancer (LC) have assessed individual factors of socioeconomic status and race/ethnicity but seldom investigate a wider breadth of SDoH-factors for their effects in the real-world. This study aims to delineate how a wider array of SDoH-vulnerabilities interactively associates with LC-disparities. MethodsThis retrospective cohort study assessed 74,495 LC-patients between 1975 and 2017 from the Surveillance-Epidemiology-End Results (SEER) database using the Social Vulnerability Index (SVI) from the CDC, total SDoH-vulnerability from 15 SDoH variables across specific vulnerabilities of socioeconomic status, minority-language status, household composition, and infrastructure/housing and transportation, which were measured across US counties. Univariate linear and logistic regressions were performed on length of care/follow-up and survival, staging, and treatment across SVI scores. ResultsSurvival time dropped significantly by 34.37% (from 72.83 to 47.80 months), and surveillance time decreased by 28.09% (from 80.99 to 58.24 months) with increasing overall social vulnerability, alongside advanced staging (OR 1.15; 95%CI 1.13–1.16), increased chemotherapy (OR 1.13; 95%CI 1.11–1.14), decreased surgical resection (OR 0.91; 95%CI 0.90–0.92), and decreased radiotherapy (OR 0.97; 95%CI 0.96–0.99). DiscussionIn this SDoH-study of LCs, detrimental care and prognostic trends were observed with increasing overall SDoH-vulnerability.more » « less
-
BackgroundThe COVID-19 pandemic has resulted in heightened levels of depression, anxiety, and other mental health issues due to sudden changes in daily life, such as economic stress, social isolation, and educational irregularity. Accurately assessing emotional and behavioral changes in response to the pandemic can be challenging, but it is essential to understand the evolving emotions, themes, and discussions surrounding the impact of COVID-19 on mental health. ObjectiveThis study aims to understand the evolving emotions and themes associated with the impact of COVID-19 on mental health support groups (eg, r/Depression and r/Anxiety) on Reddit (Reddit Inc) during the initial phase and after the peak of the pandemic using natural language processing techniques and statistical methods. MethodsThis study used data from the r/Depression and r/Anxiety Reddit communities, which consisted of posts contributed by 351,409 distinct users over a period spanning from 2019 to 2022. Topic modeling and Word2Vec embedding models were used to identify key terms associated with the targeted themes within the data set. A range of trend and thematic analysis techniques, including time-to-event analysis, heat map analysis, factor analysis, regression analysis, and k-means clustering analysis, were used to analyze the data. ResultsThe time-to-event analysis revealed that the first 28 days following a major event could be considered a critical window for mental health concerns to become more prominent. The theme trend analysis revealed key themes such as economic stress, social stress, suicide, and substance use, with varying trends and impacts in each community. The factor analysis highlighted pandemic-related stress, economic concerns, and social factors as primary themes during the analyzed period. Regression analysis showed that economic stress consistently demonstrated the strongest association with the suicide theme, whereas the substance theme had a notable association in both data sets. Finally, the k-means clustering analysis showed that in r/Depression, the number of posts related to the “depression, anxiety, and medication” cluster decreased after 2020, whereas the “social relationships and friendship” cluster showed a steady decrease. In r/Anxiety, the “general anxiety and feelings of unease” cluster peaked in April 2020 and remained high, whereas the “physical symptoms of anxiety” cluster showed a slight increase. ConclusionsThis study sheds light on the impact of COVID-19 on mental health and the related themes discussed in 2 web-based communities during the pandemic. The results offer valuable insights for developing targeted interventions and policies to support individuals and communities in similar crises.more » « less
