An official website of the United States government
Large language models (LLMs) have offered new opportunities for emotional support, and recent work has shown that they can produce empathic responses to people in distress. However, long-term mental well-being requires emotional self-regulation, where a one-time empathic response falls short. This work takes a first step by engaging with cognitive reappraisals, a strategy from psychology practitioners that uses language to targetedly change negative appraisals that an individual makes of the situation; such appraisals is known to sit at the root of human emotional experience. We hypothesize that psychologically grounded principles could enable such advanced psychology capabilities in LLMs, and design RESORT which consists of a series of reappraisal constitutions across multiple dimensions that can be used as LLM instructions. We conduct a first-of-its-kind expert evaluation (by clinical psychologists with M.S. or Ph.D. degrees) of an LLM's zero-shot ability to generate cognitive reappraisal responses to medium-length social media messages asking for support. This fine-grained evaluation showed that even LLMs at the 7B scale guided by RESORT are capable of generating empathic responses that can help users reappraise their situations.
more »
« less
Large Language Models Produce Responses Perceived to be Empathic
Large Language Models (LLMs) have demonstrated surprising performance on many tasks, including writing supportive messages that display empathy. Here, we had these models generate empathic messages in response to posts describing common life experiences, such as workplace situations, parenting, relationships, and other anxiety- and anger-eliciting situations. Across two studies (N=192, 202), we showed human raters a variety of responses written by several models (GPT4 Turbo, Llama2, and Mistral), and had people rate these responses on how empathic they seemed to be. We found that LLM-generated responses were consistently rated as more empathic than human-written responses. Linguistic analyses also show that these models write in distinct, predictable “styles”, in terms of their use of punctuation, emojis, and certain words. These results highlight the potential of using LLMs to enhance human peer support in contexts where empathy is important.
more »
« less
- Award ID(s):
- 2107524
- PAR ID:
- 10635466
- Publisher / Repository:
- IEEE
- Date Published:
- ISBN:
- 979-8-3315-1643-7
- Page Range / eLocation ID:
- 63 to 71
- Format(s):
- Medium: X
- Location:
- Glasgow, United Kingdom
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Empathy is an important skill and disposition in engineering education but measuring and assessing empathy in specific engineering contexts is a novel domain of research. In this study, we iterated on a measure of empathy in engineering design. In this refined instrument, we measured and compared responses to the same set of survey items in different configurations. In the first configuration, we measured Cognitive Empathy and Affective Empathy across three design phases. In the second configuration, we retained the focus on Cognitive Empathy and Affective Empathy and variation across three design phases, but we also differentiated between self- and other- orientated empathy. An example construct in this second configuration is Imagine-Other Cognitive Empathy in Needfinding. To provide evidence of the trustworthiness of constructs, we computed Cronbach’s alpha as a measure of internal consistency reliability and identified Spearman correlations with four extant empathy constructs as a means of external validity. All constructs in the first configuration were reliable but several constructs in the second configuration were unreliable. However, many constructs in both configurations exhibited moderate to large correlations with four existing constructs. We found students exhibited significant changes in Cognitive Empathy in Needfinding, but students did not exhibit changes in affective or cognitive empathy in other design phases. However, by employing the second configuration, we found that students demonstrated significant and positive changes in Imagine-Other Cognitive Empathy in two design phases (Concept Generation and Solution Evaluation) while exhibiting no changes in Imagine-Self Cognitive Empathy. We also analyzed students’ written responses to an open-ended question pre/post-course. This analysis revealed that, after participating in this course, students: (1) situated users as the primary rationale for design work, (2) understood addressing users’ needs as critical to design work, and (3) exhibited broadened definitions about who (or what) constitutes a user. This work provides instructors with a means to assess students’ empathy with and for users in design and to more purposefully target students’ empathic development whilst accounting for engineering design phases.more » « less
-
Empathy is an important skill and disposition in engineering education but measuring and assessing empathy in specific engineering contexts is a novel domain of research. In this study, we iterated on a measure of empathy in engineering design. In this refined instrument, we measured and compared responses to the same set of survey items in different configurations. In the first configuration, we measured Cognitive Empathy and Affective Empathy across three design phases. In the second configuration, we retained the focus on Cognitive Empathy and Affective Empathy and variation across three design phases, but we also differentiated between self- and other- orientated empathy. An example construct in this second configuration is Imagine-Other Cognitive Empathy in Needfinding. To provide evidence of the trustworthiness of constructs, we computed Cronbach’s alpha as a measure of internal consistency reliability and identified Spearman correlations with four extant empathy constructs as a means of external validity. All constructs in the first configuration were reliable but several constructs in the second configuration were unreliable. However, many constructs in both configurations exhibited moderate to large correlations with four existing constructs. We found students exhibited significant changes in Cognitive Empathy in Needfinding, but students did not exhibit changes in affective or cognitive empathy in other design phases. However, by employing the second configuration, we found that students demonstrated significant and positive changes in Imagine-Other Cognitive Empathy in two design phases (Concept Generation and Solution Evaluation) while exhibiting no changes in Imagine-Self Cognitive Empathy. We also analyzed students’ written responses to an open-ended question pre/post-course. This analysis revealed that, after participating in this course, students: (1) situated users as the primary rationale for design work, (2) understood addressing users’ needs as critical to design work, and (3) exhibited broadened definitions about who (or what) constitutes a user. This work provides instructors with a means to assess students’ empathy with and for users in design and to more purposefully target students’ empathic development whilst accounting for engineering design phases.more » « less
-
null (Ed.)Prior research indicates that empathy can help engineers achieve better outcomes in team-based, design, entrepreneurial, and humanitarian environments. We describe an educational innovation designed to teach engineering students empathic communication skills. Written in the spirit of a propagation (versus dissemination) paradigm, we focus on how the original innovation was adapted to fit into two instructional settings that differed from the first implementation context. We use first person instructor accounts to describe these adaptation processes, including interactions between the developers and the adopters of the innovation, what modifications were necessary to “fit” the innovation into the new settings, and adopter experiences. We conclude with a brief discussion of particularly salient propagation considerations that emerged for the two adopters including, for example, the amount of instructional time available for implementing the empathic communication exercises, and how to achieve student buy-in in different course settings. The two main contributions of this paper are, first, the rich descriptions of how features of the original educational innovation had to be modified to meet the two other settings’ pedagogical goals and, second, an example of how to advance scholarship that supports the propagation of engineering education teaching and learning innovations.more » « less
-
null (Ed.)One of the aims of biomedical engineering is to facilitate the development of innovative technologies to address socioeconomic challenges in healthy living and independent aging. Realizing such innovations requires empathy, agility, and creativity. This project aims to support the professional development of a competent biomedical engineer workforce that can effectively accomplish emphatic innovation, and one that can frame and re-frame problems through the innovation process. Our research examined how engineering students empathize with users and develop empathic abilities that have implications on their design innovation skills. The project team developed empathic innovation workshops and embedded them into existing biomedical engineering capstone courses. Data were collected using surveys, student project reports, ideation tasks, and observations. These workshops resulted in significant changes in students’ emphatic tendencies. From our qualitative studies, we also conjectured that the overall empathic potency of a student design team helped facilitate problem re-framing based on user input. These findings contribute to the literature on the critical role of innovation behaviors in relationship to empathic design tendencies in the context of biomedical engineering, as well as suggest instructional practices designed to promote empathy, agility, and creativity.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/ACII63134.2024.00012
