skip to main content

Explore Research Products in the NSF-PAR

Title: Sensing and Learning Human Annotators Engaged in Narrative Sensemaking
While labor issues and quality assurance in crowdwork are increasingly studied, how annotators make sense of texts and how they are personally impacted by doing so are not. We study these questions via a narrative-sorting annotation task, where carefully selected (by sequentiality, topic, emotional content, and length) collections of tweets serve as examples of everyday storytelling. As readers process these narratives, we measure their facial expressions, galvanic skin response, and self-reported reactions. From the perspective of annotator well-being, a reassuring outcome was that the sorting task did not cause a measurable stress response, however readers reacted to humor. In terms of sensemaking, readers were more confident when sorting sequential, target-topical, and highly emotional tweets. As crowdsourcing becomes more common, this research sheds light onto the perceptive capabilities and emotional impact of human readers.  more » « less
Award ID(s):
1559889
NSF-PAR ID:
10067983
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
North American Chapter of the Association for Computational Linguistics - Human Language Technology: Student Research Workshop
Page Range / eLocation ID:
136 to 143
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ( , JMIR Preprints)
    BACKGROUND

    Effective communication is crucial during health crises, and social media has become a prominent platform for public health experts to inform and to engage with the public. At the same time, social media also platforms pseudo-experts who may promote contrarian views. Despite the significance of social media, key elements of communication such as the use of moral or emotional language and messaging strategy, particularly during the COVID-19 pandemic, has not been explored.

     OBJECTIVE

    This study aims to analyze how notable public health experts (PHEs) and pseudo-experts communicated with the public during the COVID-19 pandemic. Our focus is the emotional and moral language they used in their messages across a range of pandemic issues. We also study their engagement with political elites and how the public engaged with PHEs to better understand the impact of these health experts on the public discourse.

     METHODS

    We gathered a dataset of original tweets from 489 PHEs and 356 pseudo- experts on Twitter (now X) from January 2020 to January 2021, as well as replies to the original tweets from the PHEs. We identified the key issues that PHEs and pseudo- experts prioritized. We also determined the emotional and moral language in both the original tweets and the replies. This approach enabled us to characterize key priorities for PHEs and pseudo-experts, as well as differences in messaging strategy between these two groups. We also evaluated the influence of PHE language and strategy on the public response.

     RESULTS

    Our analyses revealed that PHEs focus on masking, healthcare, education, and vaccines, whereas pseudo-experts discuss therapeutics and lockdowns more frequently. PHEs typically used positive emotional language across all issues, expressing optimism and joy. Pseudo-experts often utilized negative emotions of pessimism and disgust, while limiting positive emotional language to origins and therapeutics. Along the dimensions of moral language, PHEs and pseudo-experts differ on care versus harm, and authority versus subversion, across different issues. Negative emotional and moral language tends to boost engagement in COVID-19 discussions, across all issues. However, the use of positive language by PHEs increases the use of positive language in the public responses. PHEs act as liberal partisans: they express more positive affect in their posts directed at liberals and more negative affect directed at conservative elites. In contrast, pseudo-experts act as conservative partisans. These results provide nuanced insights into the elements that have polarized the COVID-19 discourse.

     CONCLUSIONS

    Understanding the nature of the public response to PHE’s messages on social media is essential for refining communication strategies during health crises. Our findings emphasize the need for experts to consider the strategic use of moral and emotional language in their messages to reduce polarization and enhance public trust.

     
    more » « less
  2. ; ; ; ; ( , Proceedings of the Language Resources and Evaluation Conference)
    Understanding emotions that people express during large-scale crises helps inform policy makers and first responders about the emotional states of the population as well as provide emotional support to those who need such support. We present CovidEmo, a dataset of ~3,000 English tweets labeled with emotions and temporally distributed across 18 months. Our analyses reveal the emotional toll caused by COVID-19, and changes of the social narrative and associated emotions over time. Motivated by the time-sensitive nature of crises and the cost of large-scale annotation efforts, we examine how well large pre-trained language models generalize across domains and timeline in the task of perceived emotion prediction in the context of COVID-19. Our analyses suggest that cross-domain information transfers occur, yet there are still significant gaps. We propose semi-supervised learning as a way to bridge this gap, obtaining significantly better performance using unlabeled data from the target domain. 
    more » « less
  3. ; ; ; ; ( , ACM Transactions on Computer-Human Interaction)

    Social media platforms are frequently used to share information and opinions around vaccinations. The more often a message is reshared, the wider the reach of the message and potential influence it may have on shaping people’s opinions to get vaccinated or not. We used a negative binomial regression to investigate whether a message’s linguistic characteristics (degree of concreteness, emotional arousal, and sentiment) and user characteristics (political ideology and number of followers) may influence users’ decisions to reshare tweets related to the COVID-19 vaccine. We analyzed US English-language tweets related to the COVID-19 vaccine between May 2020 and October 2021 (N = 236,054).

    Tweets with positive and high-arousal words were more often retweeted than negative, low-arousal tweets. Tweets with abstract words were more often retweeted than tweets with concrete words. In addition, while Liberal users were more likely to have tweets with a positive sentiment reshared, Conservative users were more likely to have tweets with a negative sentiment reshared. Our results can inform public health messaging on how to best phrase vaccine information to impact engagement and information resharing, and potentially persuade a wider set of people to get vaccinated.

     
    more » « less
  4. ; ; ; ; ( , Journal of Medical Internet Research)
    Background As a number of vaccines for COVID-19 are given emergency use authorization by local health agencies and are being administered in multiple countries, it is crucial to gain public trust in these vaccines to ensure herd immunity through vaccination. One way to gauge public sentiment regarding vaccines for the goal of increasing vaccination rates is by analyzing social media such as Twitter. Objective The goal of this research was to understand public sentiment toward COVID-19 vaccines by analyzing discussions about the vaccines on social media for a period of 60 days when the vaccines were started in the United States. Using the combination of topic detection and sentiment analysis, we identified different types of concerns regarding vaccines that were expressed by different groups of the public on social media. Methods To better understand public sentiment, we collected tweets for exactly 60 days starting from December 16, 2020 that contained hashtags or keywords related to COVID-19 vaccines. We detected and analyzed different topics of discussion of these tweets as well as their emotional content. Vaccine topics were identified by nonnegative matrix factorization, and emotional content was identified using the Valence Aware Dictionary and sEntiment Reasoner sentiment analysis library as well as by using sentence bidirectional encoder representations from transformer embeddings and comparing the embedding to different emotions using cosine similarity. Results After removing all duplicates and retweets, 7,948,886 tweets were collected during the 60-day time period. Topic modeling resulted in 50 topics; of those, we selected 12 topics with the highest volume of tweets for analysis. Administration and access to vaccines were some of the major concerns of the public. Additionally, we classified the tweets in each topic into 1 of the 5 emotions and found fear to be the leading emotion in the tweets, followed by joy. Conclusions This research focused not only on negative emotions that may have led to vaccine hesitancy but also on positive emotions toward the vaccine. By identifying both positive and negative emotions, we were able to identify the public's response to the vaccines overall and to news events related to the vaccines. These results are useful for developing plans for disseminating authoritative health information and for better communication to build understanding and trust. 
    more » « less
  5. ; ; ; ( , Personality and Social Psychology Bulletin)

    Although research suggests distressed individuals benefit from others’ empathy, it is unclear how an individual’s level of empathy influences dyadic responses during emotional situations. In the current study, female participants ( N = 140; 70 dyads) were paired with a stranger. One member of each dyad (the experiencer) was randomly assigned to undergo a stressful task and disclose negative personal experiences to their partner (the listener). Experiencers paired with listeners higher in dispositional emotional empathy had less negative affect during emotional disclosure and lower sympathetic nervous system reactivity during the stressful task and disclosure. Listeners higher in emotional empathy reported more negative affect in response to their partner’s distress. Furthermore, for listeners higher in emotional empathy, those who more accurately rated their partner’s emotions were more physiologically influenced by their partners. Findings shed light on interpersonal functions of empathy and suggest a stranger’s level of emotional empathy regulates distressed partner’s emotions and physiology.

     
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email