An official website of the United States government
A critical problem in deep learning is that systems learn inappropriate biases, resulting in their inability to perform well on minority groups. This has led to the creation of multiple algorithms that endeavor to mitigate bias. However, it is not clear how effective these methods are. This is because study protocols differ among papers, systems are tested on datasets that fail to test many forms of bias, and systems have access to hidden knowledge or are tuned specifically to the test set. To address this, we introduce an improved evaluation protocol, sensible metrics, and a new dataset, which enables us to ask and answer critical questions about bias mitigation algorithms. We evaluate seven state-of-the-art algorithms using the same network architecture and hyperparameter selection policy across three benchmark datasets. We introduce a new dataset called Biased MNIST that enables assessment of robustness to multiple bias sources. We use Biased MNIST and a visual question answering (VQA) benchmark to assess robustness to hidden biases. Rather than only tuning to the test set distribution, we study robustness across different tuning distributions, which is critical because for many applications the test distribution may not be known during development. We find that algorithms exploit hidden biases, are unable to scale to multiple forms of bias, and are highly sensitive to the choice of tuning set. Based on our findings, we implore the community to adopt more rigorous assessment of future bias mitigation methods. All data, code, and results are publicly available.
more »
« less
Implicit Bias and Antidiscrimination Policy
The science behind implicit bias tests (e.g., Implicit Association Test) has become the target of increased criticism. However, policymakers seeking to combat discrimination care about reducing bias in people’s actual behaviors, not about changing a person’s score on an implicit bias test. In line with this argument, we postulate that scientific controversies about implicit bias tests are irrelevant for antidiscrimination policy, which should instead focus on implicit bias in actual discriminatory behavior that occurs outside of awareness (in addition to instances of explicit bias). Two well-documented mechanisms can lead to implicit bias in actual discriminatory behavior: biased weighting and biased interpretation of information about members of particular social groups. The policy relevance of the two mechanisms is illustrated with their impact on hiring and promotion decisions, jury selection, and policing. Implications for education and bias intervention are discussed.
more »
« less
- Award ID(s):
- 1941440
- PAR ID:
- 10214668
- Date Published:
- Journal Name:
- Policy Insights from the Behavioral and Brain Sciences
- Volume:
- 7
- Issue:
- 2
- ISSN:
- 2372-7322
- Page Range / eLocation ID:
- 99 to 106
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract The aim of our study was to examine the longitudinal associations between two forms of second language (L2) knowledge (i.e., explicit and implicit knowledge) and the activity types that facilitate different processing mechanisms (i.e., form- and meaning-focused processing). L2 English speakers completed two tests of explicit knowledge (untimed written grammaticality judgment test and metalinguistic knowledge test) and three tests of implicit knowledge (timed written grammaticality judgment test, oral production, and elicited imitation) at the beginning and the end of a semester of university-level study. To track engagement in the activity types, participants completed self-reported language exposure logs across five days throughout the semester. The results from an autoregressive cross-lag analysis suggest L2 explicit and implicit knowledge influenced each other reciprocally over time. Neither activity type predicted knowledge development. We conclude that language acquisition is a developmental process typified by a dynamic, synergistic interface between explicit and implicit knowledge.more » « less
-
Predictive policing, the practice of using of algorithmic systems to forecast crime, is heralded by police departments as the new frontier of crime analysis. At the same time, it is opposed by civil rights groups, academics, and media outlets for being ‘biased’ and therefore discriminatory against communities of color. This paper argues that the prevailing focus on racial bias has overshadowed two normative factors that are essential to a full assessment of the moral permissibility of predictive policing: fairness in the social distribution of the benefits and burdens of policing as well as the distinctive role of consent in determining fair distribution. When these normative factors are given their due attention, several requirements emerge for the fair implementation of predictive policing. Among these requirements are that police departments inform and solicit buy-in from affected communities about strategic decision-making and that departments favor non-enforcement-oriented interventions.more » « less
-
Sex allocation theory predicts that females should bias their offspring sex ratios when the fitness benefits of producing sons or daughters differ depending on rearing environment. The Trivers-Willard hypothesis proposes that whether females produce more sons or daughters depends on food availability via both intrinsic maternal condition and differing reproductive potential (typically from mating system structure) for sons versus daughters. However, tests of its key predictions are often based on untested, implicit assumptions that are difficult to quantify, especially in migratory animals. In a 5-year study, we manipulated food availability in low- and high-elevation forest to test the Trivers-Willard hypothesis in the migratory black-throated blue warbler (Setophaga caerulescens). We found that the population-wide offspring sex ratio was significantly male-biased (population mean: 0.58), which was driven by an overproduction of sons in high-elevation forest (high-quality habitat mean: 0.59). Yet, we found no effect of food availability on offspring sex ratio from either natural variation or supplemental feeding. Sex-specific developmental costs did not differ for sons and daughters reared under low and high food availability. These results suggest that female black-throated blue warblers do not manipulate offspring sex ratios in response to food availability and are not consistent with the predictions of the Trivers-Willard hypothesis. This study highlights challenges of examining mechanisms driving patterns in offspring sex allocation in migratory species for which both the costs of rearing and relative fitness benefits of sons and daughters cannot be tracked into adulthood.more » « less
-
Reward learning as a method for inferring human intent and preferences has been studied extensively. Prior approaches make an implicit assumption that the human maintains a correct belief about the robot's domain dynamics. However, this may not always hold since the human's belief may be biased, which can ultimately lead to a misguided estimation of the human's intent and preferences, which is often derived from human feedback on the robot's behaviors. In this paper, we remove this restrictive assumption by considering that the human may have an inaccurate understanding of the robot. We propose a method called Generalized Reward Learning with biased beliefs about domain dynamics (GeReL) to infer both the reward function and human's belief about the robot in a Bayesian setting based on human ratings. Due to the complex forms of the posteriors, we formulate it as a variational inference problem to infer the posteriors of the parameters that govern the reward function and human's belief about the robot simultaneously. We evaluate our method in a simulated domain and with a user study where the user has a bias based on the robot's appearances. The results show that our method can recover the true human preferences while subject to such biased beliefs, in contrast to prior approaches that could have misinterpreted them completely.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1177/2372732220939128
