An official website of the United States government
To investigate the well-observed racial disparities in computer vision systems that analyze images of humans, researchers have turned to skin tone as a more objective annotation than race metadata for fairness performance evaluations. However, the current state of skin tone annotation procedures is highly varied. For instance, researchers use a range of untested scales and skin tone categories, have unclear annotation procedures, and provide inadequate analyses of uncertainty. In addition, little attention is paid to the positionality of the humans involved in the annotation process—both designers and annotators alike—and the historical and sociological context of skin tone in the United States. Our work is the first to investigate the skin tone annotation process as a sociotechnical project. We surveyed recent skin tone annotation procedures and conducted annotation experiments to examine how subjective understandings of skin tone are embedded in skin tone annotation procedures. Our systematic literature review revealed the uninterrogated association between skin tone and race and the limited effort to analyze annotator uncertainty in current procedures for skin tone annotation in computer vision evaluation. Our experiments demonstrated that design decisions in the annotation procedure such as the order in which the skin tone scale is presented or additional context in the image (i.e., presence of a face) significantly affected the resulting inter-annotator agreement and individual uncertainty of skin tone annotations. We call for greater reflexivity in the design, analysis, and documentation of procedures for evaluation using skin tone.
more »
« less
SREDS: A dichromatic separation based measure of skin color
Face recognition (FR) systems are fast becoming ubiquitous. However, differential performance among certain demographics was identified in several widely used FR models. The skin tone of the subject is an important factor in addressing the differential performance. Previous work has used modeling methods to propose skin tone measures of subjects across different illuminations or utilized subjective labels of skin color and demographic information. However, such models heavily rely on consistent background and lighting for calibration, or utilize labeled datasets, which are time-consuming to generate or are unavailable. In this work, we have developed a novel and data-driven skin color measure capable of accurately representing subjects' skin tone from a single image, without requiring a consistent background or illumination. Our measure leverages the dichromatic reflection model in RGB space to decompose skin patches into diffuse and specular bases.
more »
« less
- Award ID(s):
- 1650503
- PAR ID:
- 10318829
- Date Published:
- Journal Name:
- 2021 IEEE International Workshop on Biometrics and Forensics (IWBF)
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)Abstract The implementation of body-worn cameras (BWC) by policing agencies has received widespread support from many individuals, including citizens and police officers. Despite their increasing prevalence, little is known about how the point-of-view (POV) of these cameras affects perceptions of viewers. In this research, we investigate how POV interacts with skin color of citizens in police use of force videos to affect perceptions of procedural justice. In an experimental study, participants watched eight police use of force videos—half recorded from BWC and half from an onlooker’s perspective—in which skin tone of the citizen varied. Results indicate that POV interacts with citizen skin tone such that, compared to the onlooker perspective, the BWC exacerbated viewer racial bias against dark skin tone citizens. Furthermore, identification with the police officer fully mediated this relationship. Results are discussed in relation to media theory and practical implications.more » « less
-
As US society continues to diversify and calls for better measurements of racialized appearance increase, survey researchers need guidance about effective strategies for assessing skin color in field research. This study examined the consistency, comparability, and meaningfulness of the two most widely used skin tone rating scales (Massey–Martin and PERLA) and two portable and inexpensive handheld devices for skin color measurement (Nix colorimeter and Labby spectrophotometer). We collected data in person using these four instruments from forty-six college students selected to reflect a wide range of skin tones across four racial-ethnic groups (Asian, Black, Latinx, White). These college students, five study staff, and 459 adults from an online sample also rated forty stock photos, again selected for skin tone diversity. Our results—based on data collected under controlled conditions—demonstrate high consistency across raters and readings. The Massey–Martin and PERLA scale scores were highly linearly related to each other, although PERLA better differentiated among people with the lightest skin tones. The Nix and Labby darkness-to-lightness (L*) readings were likewise linearly related to each other and to the Massey–Martin and PERLA scores, in addition to showing expected variation within and between race ethnicities. In addition, darker Massey–Martin and PERLA ratings correlated with online raters’ expectations that a photographed person experienced greater discrimination. In contrast, the redness (a*) and yellowness (b*) undertones were highest in the mid-range of the rating scale scores and demonstrated greater overlap across race-ethnicities. Overall, each instrument showed sufficient consistency, comparability, and meaningfulness for use in field surveys when implemented soundly (e.g., not requiring memorization). However, PERLA might be preferred to Massey–Martin in studies representing individuals with the lightest skin tones, and handheld devices may be preferred to rating scales to reduce measurement error when studies could gather only a single rating.more » « less
-
Background: Color and pattern phenotypes have clear implications for survival and reproduction in many species. However, the mechanisms that produce this coloration are still poorly characterized, especially at the genomic level. Here we have taken a transcriptomics-based approach to elucidate the underlying genetic mechanisms affecting color and pattern in a highly polytypic poison frog. We sequenced RNA from the skin from four different color morphs during the final stage of metamorphosis and assembled a de novo transcriptome. We then investigated differential gene expression, with an emphasis on examining candidate color genes from other taxa. Results: Overall, we found differential expression of a suite of genes that control melanogenesis, melanocyte differentiation, and melanocyte proliferation (e.g., tyrp1, lef1, leo1, and mitf) as well as several differentially expressed genes involved in purine synthesis and iridophore development (e.g., arfgap1, arfgap2, airc, and gart). Conclusions: Our results provide evidence that several gene networks known to affect color and pattern in vertebrates play a role in color and pattern variation in this species of poison frog.more » « less
-
Machine learning (ML) based skin cancer detection tools are an example of a transformative medical technology that could potentially democratize early detection for skin cancer cases for everyone. However, due to the dependency of datasets for training, ML based skin cancer detection always suffers from a systemic racial bias. Racial communities and ethnicity not well represented within the training datasets will not be able to use these tools, leading to health disparities being amplified. Based on empirical observations we posit that skin cancer training data is biased as it’s dataset represents mostly communities of lighter skin tones, despite skin cancer being far more lethal for people of color. In this paper we use domain adaptation techniques by employing CycleGANs to mitigate racial biases existing within state of the art machine learning based skin cancer detection tools by adapting minority images to appear as the majority. Using our domain adaptation techniques to augment our minority datasets, we are able to improve the accuracy, precision, recall, and F1 score of typical image classification machine learning models for skin cancer classification from the biased 50% accuracy rate to a 79% accuracy rate when testing on minority skin tone images. We evaluate and demonstrate a proof-of-concept smartphone application.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/IWBF50991.2021.9465090
