More Like this

1. Asking Participant-specific Outcome Questions for a Deeper Understanding of Complex Events

   Vallurupalli, Sai ; Ghosh, Sayontan ; Erk, Katrin ; Balasubramian, Niranjan ; Ferraro, Francis ( December 2022 , Conference on Empirical Methods in Natural Language Processing (EMNLP))

   Knowledge about outcomes is critical for complex event understanding but is hard to acquire. We show that by pre-identifying a participant in a complex event, crowdworkers are able to (1) infer the collective impact of salient events that make up the situation, (2) annotate the volitional engagement of participants in causing the situation, and (3) ground the outcome of the situation in state changes of the participants. By creating a multi-step interface and a careful quality control strategy, we collect a high quality annotated dataset of 8K short newswire narratives and ROCStories with high inter-annotator agreement (0.74-0.96 weighted Fleiss Kappa). Our dataset, POQue (Participant Outcome Questions), enables the exploration and development of models that address multiple aspects of semantic understanding. Experimentally, we show that current language models lag behind human performance in subtle ways through our task formulations that target abstract and specific comprehension of a complex event, its outcome, and a participant’s influence over the event culmination. 

   more » « less
2. Sarcasm Detection in a Disaster Context

   Sosea, Tiberiu ; Li, Junyi Jessy ; Caragea, Cornelia ( May 2024 , Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024))

   During natural disasters, people often use social media platforms such as Twitter to ask for help, to provide information about the disaster situation, or to express contempt about the unfolding event or public policies and guidelines. This contempt is in some cases expressed as sarcasm or irony. Understanding this form of speech in a disaster-centric context is essential to improving natural language understanding of disaster-related tweets. In this paper, we introduce HurricaneSARC, a dataset of 15,000 tweets annotated for intended sarcasm, and provide a comprehensive investigation of sarcasm detection using pre-trained language models. Our best model is able to obtain as much as 0.70 F1 on our dataset. We also demonstrate that the performance on HurricaneSARC can be improved by leveraging intermediate task transfer learning 

   more » « less
3. Mining Logical Event Schemas From Pre-Trained Language Models

   https://doi.org/10.18653/v1/2022.acl-srw.25  

   Lawley, Lane ; Schubert, Lenhart ( May 2022 , Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop)

   We present NESL (the Neuro-Episodic Schema Learner), an event schema learning system that combines large language models, FrameNet parsing, a powerful logical representation of language, and a set of simple behavioral schemas meant to bootstrap the learning process. In lieu of a pre-made corpus of stories, our dataset is a continuous feed of “situation samples” from a pre-trained language model, which are then parsed into FrameNet frames, mapped into simple behavioral schemas, and combined and generalized into complex, hierarchical schemas for a variety of everyday scenarios. We show that careful sampling from the language model can help emphasize stereotypical properties of situations and de-emphasize irrelevant details, and that the resulting schemas specify situations more comprehensively than those learned by other systems. 

   more » « less
4. The “Naturalistic Free Recall” dataset: four stories, hundreds of participants, and high-fidelity transcriptions

   https://doi.org/10.1038/s41597-024-04082-6  

   Raccah, Omri ; Chen, Phoebe ; Gureckis, Todd_M ; Poeppel, David ; Vo, Vy_A ( December 2024 , Scientific Data)

   The “Naturalistic Free Recall” dataset provides transcribed verbal recollections of four spoken narratives collected from 229 participants. Each participant listened to two stories, varying in duration from approximately 8 to 13 minutes, recorded by different speakers. Subsequently, participants were tasked with verbally recalling the narrative content in as much detail as possible and in the correct order. The dataset includes high-fidelity, time-stamped text transcripts of both the original narratives and participants’ recollections. To validate the dataset, we apply a previously published automated method to score memory performance for narrative content. Using this approach, we extend effects traditionally observed in classic list-learning paradigms. The analysis of narrative contents and its verbal recollection presents unique challenges compared to controlled list-learning experiments. To facilitate the use of these rich data by the community, we offer an overview of recent computational methods that can be used to annotate and evaluate key properties of narratives and their recollections. Using advancements in machine learning and natural language processing, these methods can help the community understand the role of event structure, discourse properties, prediction error, high-level semantic features (e.g., idioms, humor), and more. All experimental materials, code, and data are publicly available to facilitate new advances in understanding human memory.

    

   more » « less
5. Modeling Social Situation Awareness in Driving Interactions

   https://doi.org/10.1145/3640792.3675717  

   Klein, Navit ; Sandhaus, Hauke ; Goedicke, David ; Ju, Wendy ; Parush, Avi ( September 2024 , ACM)

   The design of self-driving vehicles requires an understanding of the social interactions between drivers in resolving vague encounters, such as at un-signalized intersections. In this paper, we make the case for social situation awareness as a model for understanding everyday driving interaction. Using a dual-participant VR driving simulator, we collected data from driving encounter scenarios to understand how (N=170) participant drivers behave with respect to one another. Using a social situation awareness questionnaire we developed, we assessed the participants’ social awareness of other driver’s direction of approach to the intersection, and also logged signaling, speed and speed change, and heading of the vehi- cle. Drawing upon the statistically significant relationships in the variables in the study data, we propose a Social Situation Awareness model based on the approach, speed, change of speed, heading and explicit signaling from drivers. 

   more » « less