More Like this

1. Examining the Contribution of Physical Cues for Same- and Cross-Race Face Individuation

   https://doi.org/10.1177/01461672221141510  

   Correll, Joshua; Ma, Debbie_S; Kenny, David_A; Palma, Tomás_A (January 2023, Personality and Social Psychology Bulletin)

   Face individuation involves sensitivity to physical characteristics that provide information about identity. We examined whether Black and White American faces differ in terms of individuating information, and whether Black and White perceivers differentially weight information when judging same-race and cross-race faces. Study 1 analyzed 20 structural metrics (e.g., eye width, nose length) of 158 Black and White faces to determine which differentiate faces within each group. High-utility metrics (e.g., nose length, eye height, chin length) differentiated faces of both groups, low-utility metrics (e.g., face width, eye width, face length) offered less individuating information. Study 2 ( N = 4,510) explored Black and White participants’ sensitivity to variation on structural metrics using similarity ratings. High-utility metrics affected perceived dissimilarity more than low-utility metrics. This relationship was non-significantly stronger for same-race faces rather than cross-race faces. Perceivers also relied more on features that were racially stereotypic of the faces they were rating. 

   more » « less
2. Cross-Modality Protein Embedding for Compound-Protein Affinity and Contact Prediction

   You, Yuning; Shen, Yang (December 2020, Machine Learning for Structure Biology (MLSB) Workshop at the 34th Conference on Neural Information Processing Systems)

   null (Ed.)

   Compound-protein pairs dominate FDA-approved drug-target pairs and the prediction of compound-protein affinity and contact (CPAC) could help accelerate drug discovery. In this study we consider proteins as multi-modal data including 1D amino-acid sequences and (sequence-predicted) 2D residue-pair contact maps. We empirically evaluate the embeddings of the two single modalities in their accuracyand generalizability of CPAC prediction (i.e. structure-free interpretable compound-protein affinity prediction). And we rationalize their performances in both challenges of embedding individual modalities and learning generalizable embedding-label relationship. We further propose two models involving cross-modality protein embedding and establish that the one with cross interaction (thus capturing correlations among modalities) outperforms SOTAs and our single modality models in affinity, contact, and binding-site predictions for proteins never seen in the training set. 

   more » « less
3. Within-person dynamics of attention to race and expression of race bias: a real-time test of the self-regulation of prejudice model

   https://doi.org/10.1093/scan/nsaf026  

   Brancaleone, Paul_J; Cofresí, Roberto_U; Volpert-Esmond, Hannah_I; Amodio, David_M; Ito, Tiffany_A; Bartholow, Bruce_D (March 2025, Social Cognitive and Affective Neuroscience)

   Abstract Prior research has supported some aspects of a theorized prejudice self-regulation model. We provide the first test of the full model-based process of bias regulation as it unfolds in real time. Event-related potentials (ERPs) were recorded from White undergraduates at two large American universities (N = 130; 40% female) during a racial stereotype priming task. Attention to Black male face primes, indexed by the P2 ERP, increased following self-regulation failures. In turn, within-person, trial-to-trial variability in attention to Black male faces predicted variability in bias expression. The latter effect was moderated by individual differences in internal motivation to respond without prejudice (IMS). Specifically, among lower-IMS individuals, trials in which Black faces elicited relatively larger P2 amplitudes (relative to an individual’s own average P2 amplitude) were associated with increased behavioral race bias. In contrast, and consistent with theory, among higher-IMS individuals trials in which Black faces elicited larger relative P2 amplitudes were associated with decreased bias. Findings provide direct evidence supporting the temporal sequencing of race-bias regulation and identify within-person variability in attention to race as a potential mechanism for determining when and in whom bias will be regulated. 

   more » « less
4. Inclusive Portraits: Race-Aware Human-in-the-Loop Technology

   https://doi.org/10.1145/3617694.3623235  

   Flores-Saviaga, Claudia; Curtis, Christopher; Savage, Saiph (October 2023, ACM)

   AI has revolutionized the processing of various services, including the automatic facial verification of people. Automated approaches have demonstrated their speed and efficiency in verifying a large volume of faces, but they can face challenges when processing content from certain communities, including communities of people of color. This challenge has prompted the adoption of "human-inthe-loop" (HITL) approaches, where human workers collaborate with the AI to minimize errors. However, most HITL approaches do not consider workers’ individual characteristics and backgrounds. This paper proposes a new approach, called Inclusive Portraits (IP), that connects with social theories around race to design a racially-aware human-in-the-loop system. Our experiments have provided evidence that incorporating race into human-in-the-loop (HITL) systems for facial verification can significantly enhance performance, especially for services delivered to people of color. Our findings also highlight the importance of considering individual worker characteristics in the design of HITL systems, rather than treating workers as a homogenous group. Our research has significant design implications for developing AI-enhanced services that are more inclusive and equitable. 

   more » « less
5. FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age for Bias Measurement and Mitigation

   https://doi.org/10.1109/WACV48630.2021.00159  

   Karkkainen, Kimmo; Joo, Jungseock (January 2021, 2021 IEEE Winter Conference on Applications of Computer Vision (WACV))

   null (Ed.)

   Existing public face image datasets are strongly biased toward Caucasian faces, and other races (e.g., Latino) are significantly underrepresented. The models trained from such datasets suffer from inconsistent classification accuracy, which limits the applicability of face analytic systems to non-White race groups. To mitigate the race bias problem in these datasets, we constructed a novel face image dataset containing 108,501 images which is balanced on race. We define 7 race groups: White, Black, Indian, East Asian, Southeast Asian, Middle Eastern, and Latino. Images were collected from the YFCC-100M Flickr dataset and labeled with race, gender, and age groups. Evaluations were performed on existing face attribute datasets as well as novel image datasets to measure the generalization performance. We find that the model trained from our dataset is substantially more accurate on novel datasets and the accuracy is consistent across race and gender groups. We also compare several commercial computer vision APIs and report their balanced accuracy across gender, race, and age groups. Our code, data, and models are available at https://github.com/joojs/fairface. 

   more » « less