skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Reducing subgroup differences in personnel selection through the application of machine learning
Abstract Researchers have investigated whether machine learning (ML) may be able to resolve one of the most fundamental concerns in personnel selection, which is by helping reduce the subgroup differences (and resulting adverse impact) by race and gender in selection procedure scores. This article presents three such investigations. The findings show that the growing practice of making statistical adjustments to (nonlinear) ML algorithms to reduce subgroup differences must create predictive bias (differential prediction) as a mathematical certainty. This may reduce validity and inadvertently penalize high‐scoring racial minorities. Similarly, one approach that adjusts the ML input data only slightly reduces the subgroup differences but at the cost of slightly reduced model accuracy. Other emerging tactics involve weighting predictors to balance or find a compromise between the competing goals of reducing subgroup differences while maintaining validity, but they have been limited to two outcomes. The third investigation extends this to three outcomes (e.g., validity, subgroup differences, and cost) and presents an online tool. Collectively, the studies in this article illustrate that ML is unlikely to be able to resolve the issue of adverse impact, but it may assist in finding incremental improvements.  more » « less
Award ID(s):
2309853 2040807
PAR ID:
10426679
Author(s) / Creator(s):
 ;  ;  ;  ;  ;  ;  ;  ;  ;  ;  ;  ;  
Publisher / Repository:
Wiley-Blackwell
Date Published:
Journal Name:
Personnel Psychology
Volume:
76
Issue:
4
ISSN:
0031-5826
Format(s):
Medium: X Size: p. 1125-1159
Size(s):
p. 1125-1159
Sponsoring Org:
National Science Foundation
More Like this
  1. ABSTRACT Maladapted immigrants may reduce wild population productivity and resilience, depending on the degree of fitness mismatch between dispersers and locals. Thus, domesticated individuals escaping into wild populations is a key conservation concern. In Prince William Sound, Alaska, over 700 million pink salmon (Oncorhynchus gorbuscha) are released annually from hatcheries, providing a natural experiment to characterize the mechanisms underlying impacts to wild populations. Using a dataset of > 200,000 pink salmon sampled from 30 populations over 8 years, we detected significant body size and phenological differences between hatchery‐ and wild‐origin spawners, likely driven by competitive differences during maturation and broodstock selection practices. Variation in traits was reduced in hatchery fish, raising biodiversity concerns. However, phenotypic traits of immigrants and locals were positively correlated. We discuss possible mechanisms that may explain this pattern and how it may reduce adverse impacts associated with reduced trait variation. This study suggests that domestication impacts are likely widespread, but local adaptation may be maintained by phenotypic sorting. 
    more » « less
  2. Abstract As an increasing number of machine learning (ML) products enter the research-to-operations (R2O) pipeline, researchers have anecdotally noted a perceived hesitancy by operational forecasters to adopt this relatively new technology. One explanation often cited in the literature is that this perceived hesitancy derives from the complex and opaque nature of ML methods. Because modern ML models are trained to solve tasks by optimizing a potentially complex combination of mathematical weights, thresholds, and nonlinear cost functions, it can be difficult to determine how these models reach a solution from their given input. However, it remains unclear to what degree a model’s transparency may influence a forecaster’s decision to use that model or if that impact differs between ML and more traditional (i.e., non-ML) methods. To address this question, a survey was offered to forecaster and researcher participants attending the 2021 NOAA Hazardous Weather Testbed (HWT) Spring Forecasting Experiment (SFE) with questions about how participants subjectively perceive and compare machine learning products to more traditionally derived products. Results from this study revealed few differences in how participants evaluated machine learning products compared to other types of guidance. However, comparing the responses between operational forecasters, researchers, and academics exposed notable differences in what factors the three groups considered to be most important for determining the operational success of a new forecast product. These results support the need for increased collaboration between the operational and research communities. Significance StatementParticipants of the 2021 Hazardous Weather Testbed Spring Forecasting Experiment were surveyed to assess how machine learning products are perceived and evaluated in operational settings. The results revealed little difference in how machine learning products are evaluated compared to more traditional methods but emphasized the need for explainable product behavior and comprehensive end-user training. 
    more » « less
  3. Myocardial infarctions (MIs) kickstart an intense inflammatory response resulting in extracellular matrix (ECM) degradation, wall thinning, and chamber dilation that leaves the heart susceptible to rupture. Reperfusion therapy is one of the most effective strategies for limiting adverse effects of MIs, but is a challenge to administer in a timely manner. Late reperfusion therapy (LRT; 3 + hours post-MI) does not limit infarct size, but does reduce incidences of post-MI rupture and improves long-term patient outcomes. Foundational studies employing LRT in the mid-twentieth century revealed beneficial reductions in infarct expansion, aneurysm formation, and left ventricle dysfunction. The mechanism by which LRT acts, however, is undefined. Structural analyses, relying largely on one-dimensional estimates of ECM composition, have found few differences in collagen content between LRT and permanently occluded animal models when using homogeneous samples from infarct cores. Uniaxial testing, on the other hand, revealed slight reductions in stiffness early in inflammation, followed soon after by an enhanced resistance to failure for cases of LRT. The use of one-dimensional estimates of ECM organization and gross mechanical function have resulted in a poor understanding of the infarct’s spatially variable mechanical and structural anisotropy. To resolve these gaps in literature, future work employing full-field mechanical, structural, and cellular analyses is needed to better define the spatiotemporal post-MI alterations occurring during the inflammatory phase of healing and how they are impacted following reperfusion therapy. In turn, these studies may reveal how LRT affects the likelihood of rupture and inspire novel approaches to guide scar formation. 
    more » « less
  4. This article presents new quasi-experimental evidence regarding the effectiveness of teaching-oriented faculty with tenure-track appointment, a model pioneered at the University of California (UC) system. Using data from six cohorts of students at a UC campus, we examine the impact of initial course-taking with three distinct types of instructors—tenure-track research faculty, tenure-track teaching faculty, and contingent lecturers—on students’ current and subsequent academic outcomes. Descriptive analyses indicate that tenure-track teaching faculty assume a substantially larger teaching load than either research faculty or lecturers. Using a three-way fixed effects model, we find limited evidence supporting differences by faculty type on either current or downstream student outcomes. 
    more » « less
  5. This article presents a type-based analysis for deriving upper bounds on the expected execution cost of probabilistic programs. The analysis is naturally compositional, parametric in the cost model, and supports higher-order functions and inductive data types. The derived bounds are multivariate polynomials that are functions of data structures. Bound inference is enabled by local type rules that reduce type inference to linear constraint solving. The type system is based on the potential method of amortized analysis and extends automatic amortized resource analysis (AARA) for deterministic programs. A main innovation is that bounds can contain symbolic probabilities, which may appear in data structures and function arguments. Another contribution is a novel soundness proof that establishes the correctness of the derived bounds with respect to a distribution-based operational cost semantics that also includes nontrivial diverging behavior. For cost models like time, derived bounds imply termination with probability one. To highlight the novel ideas, the presentation focuses on linear potential and a core language. However, the analysis is implemented as an extension of Resource Aware ML and supports polynomial bounds and user defined data structures. The effectiveness of the technique is evaluated by analyzing the sample complexity of discrete distributions and with a novel average-case estimation for deterministic programs that combines expected cost analysis with statistical methods. 
    more » « less