skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: A PAC Framework for Aggregating Agents’ Judgments
Specifying the objective function that an AI system should pursue can be challenging. Especially when the decisions to be made by the system have a moral component, input from multiple stakeholders is often required. We consider approaches that query them about their judgments in individual examples, and then aggregate these judgments into a general policy. We propose a formal learning-theoretic framework for this setting. We then give general results on how to translate classical results from PAC learning into results in our framework. Subsequently, we show that in some settings, better results can be obtained by working directly in our framework. Finally, we discuss how our model can be extended in a variety of ways for future research.  more » « less
Award ID(s):
1814056
PAR ID:
10196239
Author(s) / Creator(s):
;
Date Published:
Journal Name:
Proceedings of the AAAI Conference on Artificial Intelligence
Volume:
33
ISSN:
2159-5399
Page Range / eLocation ID:
2237 to 2244
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; (, Transactions of the Association for Computational Linguistics)
    Abstract Social stereotypes negatively impact individuals’ judgments about different groups and may have a critical role in understanding language directed toward marginalized groups. Here, we assess the role of social stereotypes in the automated detection of hate speech in the English language by examining the impact of social stereotypes on annotation behaviors, annotated datasets, and hate speech classifiers. Specifically, we first investigate the impact of novice annotators’ stereotypes on their hate-speech-annotation behavior. Then, we examine the effect of normative stereotypes in language on the aggregated annotators’ judgments in a large annotated corpus. Finally, we demonstrate how normative stereotypes embedded in language resources are associated with systematic prediction errors in a hate-speech classifier. The results demonstrate that hate-speech classifiers reflect social stereotypes against marginalized groups, which can perpetuate social inequalities when propagated at scale. This framework, combining social-psychological and computational-linguistic methods, provides insights into sources of bias in hate-speech moderation, informing ongoing debates regarding machine learning fairness. 
    more » « less
  2. ; ; ; ; (, Psychological Science)
    People make subjective judgments about the healthiness of different foods every day, and these judgments in turn influence their food choices and health outcomes. Despite the importance of such judgments, there are few quantitative theories about their psychological underpinnings. This article introduces a novel computational approach that can approximate people’s knowledge representations for thousands of common foods. We used these representations to predict how both lay decision-makers (the general population) and experts judge the healthiness of individual foods. We also applied our method to predict the impact of behavioral interventions, such as the provision of front-of-pack nutrient and calorie information. Across multiple studies with data from 846 adults, our models achieved very high accuracy rates ( r 2 = .65–.77) and significantly outperformed competing models based on factual nutritional content. These results illustrate how new computational methods applied to established psychological theory can be used to better predict, understand, and influence health behavior. 
    more » « less
  3. (, Developmental Review)
    There is a long-standing interest in the role that children’s understanding of pretense plays in their more general theory of mind development. Some argue that children understand pretense as a mental state, and the capacity to pretend is indicative of children possessing the capacity for mental representations. Others argue that children understand pretense in terms of actions and appearances, and an understanding of the mental states involved in pretending has a prolonged developmental trajectory. The goal of this paper is to integrate these ideas by positing that children understand pretense as a form of causal inference, which is based on both their general causal reasoning capacities and specific knowledge of mental states. I will first review literature on children’s understanding of pretense, and how such understanding can be conceptualized as integrating with children’s causal reasoning ability. I will then consider how children’s causal knowledge affects the ways they make inferences about others’ pretense. Next, I will consider the role of causal knowledge more broadly in children’s reasoning about pretense worlds, judgments of possibility, and counterfactual reasoning. Taken together the goal of this review is to synthesize how children understand pretending into a rational constructivist framework for understanding social cognitive development in a more integrative manner. 
    more » « less
  4. ; ; ; ; (, American Society for Engineering Education (ASEE) PEER)
    As engineering students transition from their undergraduate education into their first full-time roles within industry, they are often facing a two year induction period as a result of a Theory-to-Practice gap (Gao & Rhinehart, 2004; Rhinehart, 2019, Rhinehart 2015). The gap between engineering students and industry practitioners can be the result of many different factors: students learning complex and fundamental concepts through simple problems, students having difficulty combining knowledge from different courses to solve realistic scenarios, or the lack of time students have to master these concepts (Rhinehart, 2015). This two year induction period causes problems for the company, the individuals, and for higher education, so it is important to identify areas where this gap exists and how it can potentially be mitigated. One area worthy of investigation related to the Theory to Practice gap is the field of process safety education due to its significant impact on professional practice. This pilot study sought to gain an initial understanding of what differences may exist between how experienced industry practitioners and undergraduate engineering students approach process safety judgments. We used this data as a means for determining if approaches to process safety judgments may be an area related to where this gap has been observed. As part of the pilot study, we conducted interviews with both students and practitioners where we provided them with a list of competing criteria that are relevant to process safety judgements such as time, production, and relationships, and then asked them to describe their approach to making process safety judgments given five specific scenarios. We found that industry practitioners and students were both relying on previous experience when describing their approaches to process safety judgments. Practitioners related the scenarios to prior work place events, while students connected them to problems they learned about in class, internships, or retail jobs. A noted difference between industry practitioners and students was that industry practitioners also described being heavily influenced by relationships with co-workers, superiors, and families when approaching these judgments, which seemed to be lacking in the student responses. Past process safety incidents, as documented by the United States Chemical Safety and Hazard Investigation Board (CSB), have shown that the dynamics of relationships can have an impact on judgment processes which lead to detrimental results. The findings from this study provide additional support for the role of relationships in process safety judgments and the need for process safety instruction that addresses this role. Moving forward, it will be important to expose undergraduate students to the role of relationships in judgment-making processes so that we can better prepare them to navigate the complexities of process safety judgments. 
    more » « less
  5. ; ; ; ; (, American Society for Engineering Education (ASEE) PEER)
    As engineering students transition from their undergraduate education into their first full-time roles within industry, they are often facing a two year induction period as a result of a Theory-to-Practice gap (Gao & Rhinehart, 2004; Rhinehart, 2019, Rhinehart 2015). The gap between engineering students and industry practitioners can be the result of many different factors: students learning complex and fundamental concepts through simple problems, students having difficulty combining knowledge from different courses to solve realistic scenarios, or the lack of time students have to master these concepts (Rhinehart, 2015). This two year induction period causes problems for the company, the individuals, and for higher education, so it is important to identify areas where this gap exists and how it can potentially be mitigated. One area worthy of investigation related to the Theory to Practice gap is the field of process safety education due to its significant impact on professional practice. This pilot study sought to gain an initial understanding of what differences may exist between how experienced industry practitioners and undergraduate engineering students approach process safety judgments. We used this data as a means for determining if approaches to process safety judgments may be an area related to where this gap has been observed. As part of the pilot study, we conducted interviews with both students and practitioners where we provided them with a list of competing criteria that are relevant to process safety judgements such as time, production, and relationships, and then asked them to describe their approach to making process safety judgments given five specific scenarios. We found that industry practitioners and students were both relying on previous experience when describing their approaches to process safety judgments. Practitioners related the scenarios to prior work place events, while students connected them to problems they learned about in class, internships, or retail jobs. A noted difference between industry practitioners and students was that industry practitioners also described being heavily influenced by relationships with co-workers, superiors, and families when approaching these judgments, which seemed to be lacking in the student responses. Past process safety incidents, as documented by the United States Chemical Safety and Hazard Investigation Board (CSB), have shown that the dynamics of relationships can have an impact on judgment processes which lead to detrimental results. The findings from this study provide additional support for the role of relationships in process safety judgments and the need for process safety instruction that addresses this role. Moving forward, it will be important to expose undergraduate students to the role of relationships in judgment-making processes so that we can better prepare them to navigate the complexities of process safety judgments. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email