skip to main content

Explore Research Products in the NSF-PAR

Title: An Auditing Imperative for Automated Hiring
The high bar of proof to demonstrate either a disparate treatment or disparate impact cause of action under Title VII of the Civil Rights Act, coupled with the “black box” nature of many automated hiring systems, renders the detection and redress of bias in such algorithmic systems difficult. This Article, with contributions at the intersection of administrative law, employment & labor law, and law & technology, makes the central claim that the automation of hiring both facilitates and obfuscates employment discrimination. That phenomenon and the deployment of intellectual property law as a shield against the scrutiny of automated systems combine to form an insurmountable obstacle for disparate impact claimants. To ensure against the identified “bias in, bias out” phenomenon associated with automated decision-making, I argue that the employer’s affirmative duty of care as posited by other legal scholars creates “an auditing imperative” for algorithmic hiring systems. This auditing imperative mandates both internal and external audits of automated hiring systems, as well as record-keeping initiatives for job applications. Such audit requirements have precedent in other areas of law, as they are not dissimilar to the Occupational Safety and Health Administration (OSHA) audits in labor law or the Sarbanes-Oxley Act audit requirements in securities law. I also propose that employers that have subjected their automated hiring platforms to external audits could receive a certification mark, “the Fair Automated Hiring Mark,” which would serve to positively distinguish them in the labor market. Labor law mechanisms such as collective bargaining could be an effective approach to combating the bias in automated hiring by establishing criteria for the data deployed in automated employment decision-making and creating standards for the protection and portability of said data. The Article concludes by noting that automated hiring, which captures a vast array of applicant data, merits greater legal oversight given the potential for “algorithmic blackballing,” a phenomenon that could continue to thwart many applicants’ future job bids.  more » « less
Award ID(s):
2125174
NSF-PAR ID:
10267580
Author(s) / Creator(s):
Date Published:
Journal Name:
Harvard journal of law technology
ISSN:
2153-263X
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ( , Cardozo law review)
    null (Ed.)
    A received wisdom is that automated decision-making serves as an anti-bias intervention. The conceit is that removing humans from the decision-making process will also eliminate human bias. The paradox, however, is that in some instances, automated decision-making has served to replicate and amplify bias. With a case study of the algorithmic capture of hiring as heuristic device, this Article provides a taxonomy of problematic features associated with algorithmic decision-making as anti-bias intervention and argues that those features are at odds with the fundamental principle of equal opportunity in employment. To examine these problematic features within the context of algorithmic hiring and to explore potential legal approaches to rectifying them, the Article brings together two streams of legal scholarship: law and technology studies and employment & labor law. Counterintuitively, the Article contends that the framing of algorithmic bias as a technical problem is misguided. Rather, the Article’s central claim is that bias is introduced in the hiring process, in large part, due to an American legal tradition of deference to employers, especially allowing for such nebulous hiring criterion as “cultural fit.” The Article observes the lack of legal frameworks that take into account the emerging technological capabilities of hiring tools which make it difficult to detect disparate impact. The Article thus argues for a re-thinking of legal frameworks that take into account both the liability of employers and those of the makers of algorithmic hiring systems who, as brokers, owe a fiduciary duty of care. Particularly related to Title VII, the Article proposes that in legal reasoning corollary to extant tort doctrines, an employer’s failure to audit and correct its automated hiring platforms for disparate impact could serve as prima facie evidence of discriminatory intent, leading to the development of the doctrine of discrimination per se. The article also considers other approaches separate from employment law such as establishing consumer legal protections for job applicants that would mandate their access to the dossier of information consulted by automated hiring systems in making the employment decision. 
    more » « less
  2. ; ; ; ; ; ; ( , AIES '22: Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society)
    Automated hiring systems are among the fastest-developing of all high-stakes AI systems. Among these are algorithmic personality tests that use insights from psychometric testing, and promise to surface personality traits indicative of future success based on job seekers' resumes or social media profiles. We interrogate the reliability of such systems using stability of the outputs they produce, noting that reliability is a necessary, but not a sufficient, condition for validity. We develop a methodology for an external audit of stability of algorithmic personality tests, and instantiate this methodology in an audit of two systems, Humantic AI and Crystal. Rather than challenging or affirming the assumptions made in psychometric testing --- that personality traits are meaningful and measurable constructs, and that they are indicative of future success on the job --- we frame our methodology around testing the underlying assumptions made by the vendors of the algorithmic personality tests themselves. In our audit of Humantic AI and Crystal, we find that both systems show substantial instability on key facets of measurement, and so cannot be considered valid testing instruments. For example, Crystal frequently computes different personality scores if the same resume is given in PDF vs. in raw text, violating the assumption that the output of an algorithmic personality test is stable across job-irrelevant input variations. Among other notable findings is evidence of persistent --- and often incorrect --- data linkage by Humantic AI. An open-source implementation of our auditing methodology, and of the audits of Humantic AI and Crystal, is available at https://github.com/DataResponsibly/hiring-stability-audit. 
    more » « less
  3. ; ; ; ; ; ; ; ( , Data Mining and Knowledge Discovery)
    Abstract

    Automated hiring systems are among the fastest-developing of all high-stakes AI systems. Among these are algorithmic personality tests that use insights from psychometric testing, and promise to surface personality traits indicative of future success based on job seekers’ resumes or social media profiles. We interrogate the validity of such systems using stability of the outputs they produce, noting that reliability is a necessary, but not a sufficient, condition for validity. Crucially, rather than challenging or affirming the assumptions made in psychometric testing — that personality is a meaningful and measurable construct, and that personality traits are indicative of future success on the job — we frame our audit methodology around testing the underlying assumptions made by the vendors of the algorithmic personality tests themselves. Our main contribution is the development of a socio-technical framework for auditing the stability of algorithmic systems. This contribution is supplemented with an open-source software library that implements the technical components of the audit, and can be used to conduct similar stability audits of algorithmic systems. We instantiate our framework with the audit of two real-world personality prediction systems, namely, Humantic AI and Crystal. The application of our audit framework demonstrates that both these systems show substantial instability with respect to key facets of measurement, and hence cannot be considered valid testing instruments.

     
    more » « less
  4. ; ; ( , Work and Occupations)
    In this article, we ask whether macro-level changes during the first year of the COVID-19 pandemic relate to changes in the levels of discrimination against women and Black job-seekers at the point of hire. We develop three main hypotheses: that discrimination against women and Black job-seekers increases due to a reduction in labor demand; that discrimination against women decreases due to the reduced supply of women employees and applicants; and that discrimination against Black job-seekers decreases due to increased attention toward racial inequities associated with the Black Lives Matter protests during the summer of 2020. We test these hypotheses using a correspondence audit study collected over two periods, before and during the early COVID-19 pandemic, for one professional occupation: accountants. We find that White women experience a positive change in callbacks during the pandemic, being preferred over White men, and this change is concentrated in geographic areas that experienced relatively larger decreases in women's labor supply. Black women experience discrimination pre-pandemic but receive similar callbacks to White men during the pandemic. In contrast to both White and Black women, discrimination against Black men is persistent before and during the pandemic. Our findings are consistent with the prediction of gender-specific changes in labor supply being associated with gender-specific changes in hiring discrimination during the COVID-19 pandemic. More broadly, our study shows how hiring decision-making is related to macro-level labor market processes. 
    more » « less
  5. ; ( , Proceedings of the ... International Florida Artificial Intelligence Research Society Conference)
    Algorithmic auditing has emerged as an important methodology that gleans insights from opaque platform algorithms. These audits often rely on the repeated observations of an algorithm’s outputs given a fixed set of inputs. For example, to audit Google search, one repeatedly inputs queries and captures the resulting search pages. Then, the goal is to uncover patterns in the data that reveal the “secrets” of algorithmic decision making. In this paper, we introduce one particular algorithm audit, that of Google’s Top stories. We describe the process of data collection, exploration, and analysis for this application and share some of the insights. Concretely, our analysis suggests that Google may be trying to burst the “filter bubble” by choosing less known publishers for the 3rd position in the Top stories. In addition to revealing the behavior of the platform, the audit also illustrated that a subset of publishers cover certain stories more than others. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email