More Like this

1. Russian PropBank

   Moeller, Sarah; Wagner, Irina; Palmer, Martha; Conger, Kathryn; Myers, Skatje (May 2020, Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020),)

   This paper presents a proposition bank for Russian (RuPB), a resource for semantic role labeling (SRL). The motivating goal for this resource is to automatically project semantic role labels from English to Russian. This paper describes frame creation strategies, coverage, and the process of sense disambiguation. It discusses language-specific issues that complicated the process of building the PropBank and how these challenges were exploited as language-internal guidance for consistency and coherence. 

   more » « less
2. Adjudicating LLMs as PropBank Adjudicators

   Bonn, Julia; Madabushi, Harish Tayyar; Hwang, Jena D; Bonial, Claire (May 2024, ELRA and ICCL)

   Bonial, Claire; Bonn, Julia; Hwang, Jena D (Ed.)

   We evaluate the ability of large language models (LLMs) to provide PropBank semantic role label annotations across different realizations of the same verbs in transitive, intransitive, and middle voice constructions. In order to assess the meta-linguistic capabilities of LLMs as well as their ability to glean such capabilities through in-context learning, we evaluate the models in a zero-shot setting, in a setting where it is given three examples of another verb used in transitive, intransitive, and middle voice constructions, and finally in a setting where it is given the examples as well as the correct sense and roleset information. We find that zero-shot knowledge of PropBank annotation is almost nonexistent. The largest model evaluated, GPT-4, achieves the best performance in the setting where it is given both examples and the correct roleset in the prompt, demonstrating that larger models can ascertain some meta-linguistic capabilities through in-context learning. However, even in this setting, which is simpler than the task of a human in PropBank annotation, the model achieves only 48% accuracy in marking numbered arguments correctly. To ensure transparency and reproducibility, we publicly release our dataset and model responses. 

   more » « less
3. PropBank goes Public: Incorporation into Wikidata

   Spaulding, E; Conger, K; Gershman, A; Morshed, M; Brown, S; Pustejovsky, J; Uceda-Sosa, R; Ge, S; Palmer, M (March 2024, Association for Computational Linguistics)

   Henning, S; Stede, M (Ed.)

   This paper presents the first integration of PropBank role information into Wikidata, in order to provide a novel resource for information extraction, one combining Wikidata`s ontological metadata with PropBank`s rich argument structure encoding for event classes. We discuss a technique for PropBank augmentation to existing eventive Wikidata items, as well as identification of gaps in Wikidata`s coverage based on manual examination of over 11,300 PropBank rolesets. We propose five new Wikidata properties to integrate PropBank structure into Wikidata so that the annotated mappings can be added en masse. We then outline the methodology and challenges of this integration, including annotation with the combined resources. 

   more » « less
4. PropBank Comes of Age—Larger, Smarter, and more Diverse

   https://doi.org/10.18653/v1/2022.starsem-1.24  

   Pradhan, Sameer; Bonn, Julia; Myers, Skatje; Conger, Kathryn; O’gorman, Tim; Gung, James; Wright-bettner, Kristin; Palmer, Martha (July 2022, Proceedings of the 11th Joint Conference on Lexical and Computational Semantics)

   Vivi Nastase; Ellie Pavlick; Mohammad Taher Pilehvar; Jose Camacho-Collados; Alessandro Raganato (Ed.)

   This paper describes the evolution of the PropBank approach to semantic role labeling over the last two decades. During this time the PropBank frame files have been expanded to include non-verbal predicates such as adjectives, prepositions and multi-word expressions. The number of domains, genres and languages that have been PropBanked has also expanded greatly, creating an opportunity for much more challenging and robust testing of the generalization capabilities of PropBank semantic role labeling systems. We also describe the substantial effort that has gone into ensuring the consistency and reliability of the various annotated datasets and resources, to better support the training and evaluation of such systems 

   more » « less
5. ClearTAC: Verb Tense, Aspect, and Form Classification Using Neural Nets

   Myers, Skatje; Palmer, Martha (July 2019, Proceedings of the 1st Designing Meaning Representations Workshop, DMR-2019, held in conjunction with ACL,)

   This paper proposes using a Bidirectional LSTM-CRF model in order to identify the tense and aspect of verbs. The information that this classifier outputs can be useful for ordering events and can provide a pre-processing step to improve efficiency of annotating this type of information. This neural network architecture has been successfully employed for other sequential labeling tasks, and we show that it significantly outperforms the rule-based tool TMV-annotator on the PropBank I dataset. 

   more » « less