An official website of the United States government
While advances in fairness and alignment have helped mitigate overt biases exhibited by large language models (LLMs) when explicitly prompted, we hypothesize that these models may still exhibit implicit biases when simulating human behavior. To test this hypothesis, we propose a technique to systematically uncover such biases across a broad range of sociodemographic categories by assessing decision-making disparities among agents with LLM-generated, sociodemographically-informed personas. Using our technique, we tested six LLMs across three sociodemographic groups and four decision-making scenarios. Our results show that state-of-the-art LLMs exhibit significant sociodemographic disparities in nearly all simulations, with more advanced models exhibiting greater implicit biases despite reducing explicit biases. Furthermore, when comparing our findings to real-world disparities reported in empirical studies, we find that the biases we uncovered are directionally aligned but markedly amplified. This directional alignment highlights the utility of our technique in uncovering systematic biases in LLMs rather than random variations; moreover, the presence and amplification of implicit biases emphasizes the need for novel strategies to address these biases.
more »
« less
This content will become publicly available on October 1, 2026
Incorporating Language Technologies and LLMs to Support Breast Cancer Education in Hispanic Populations: A Web-Based, Interactive Platform
Breast cancer is a leading cause of mortality among women, disproportionately affecting Hispanic populations in the U.S., particularly those with limited health literacy and language access. To address these disparities, we present a bilingual, web-based educational platform tailored to low-literacy Hispanic users. The platform supports full navigation in English and Spanish, with seamless language switching and both written and spoken input options. It incorporates automatic speech recognition (ASR) capable of handling code-switching, enhancing accessibility for bilingual users. Educational content is delivered through culturally sensitive videos organized into four categories: prevention, detection, diagnosis, and treatment. Each video includes embedded and post-video assessment questions aligned with Bloom’s Taxonomy to foster active learning. Users can monitor their progress and quiz performance via a personalized dashboard. An integrated chatbot, powered by large language models (LLMs), allows users to ask foundational breast cancer questions in natural language. The platform also recommends relevant resources, including nearby treatment centers, and support groups. LLMs are further used for ASR, question generation, and semantic response evaluation. Combining language technologies and LLMs reduces disparities in cancer education and supports informed decision-making among underserved populations, playing a pivotal role in reducing information gaps and promoting informed healthcare decisions.
more »
« less
- Award ID(s):
- 2219587
- PAR ID:
- 10655808
- Publisher / Repository:
- MDPI
- Date Published:
- Journal Name:
- Applied Sciences
- Volume:
- 15
- Issue:
- 20
- ISSN:
- 2076-3417
- Page Range / eLocation ID:
- 11231
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Mathematics literacy is crucial in many STEM fields, yet Black and Hispanic students are less likely to achieve high math proficiency. While previous literature investigated potential factors to mitigate racial or gendered disparities in mathematics literacy, few studies attended to the conditions under which the causal interpretation of the results obtained can be established. Guided by intersectionality theory and causal decomposition analysis, we examined the degree to which disparities in mathematics literacy (a) exist at the intersection of race and gender and (b) can be reduced by hypothetical interventions that equalize school socioeconomic status (SES) or opportunity to learn (OTL) across groups. We found large racial/ethnic differences in math literacy favoring Asians and whites and much smaller gender differences. We also found that equalizing school SES may reduce disparities for Black and Hispanic males and females; equalizing OTL may reduce disparities for Black and Hispanic males as well as Asian males and females; compared to white males. Our findings suggest that interventions that target specific race–gender groups are required to reduce disparities in math literacy.more » « less
-
Opening a conversation on responsible environmental data science in the age of large language modelsAbstract The general public and scientific community alike are abuzz over the release of ChatGPT and GPT-4. Among many concerns being raised about the emergence and widespread use of tools based on large language models (LLMs) is the potential for them to propagate biases and inequities. We hope to open a conversation within the environmental data science community to encourage the circumspect and responsible use of LLMs. Here, we pose a series of questions aimed at fostering discussion and initiating a larger dialogue. To improve literacy on these tools, we provide background information on the LLMs that underpin tools like ChatGPT. We identify key areas in research and teaching in environmental data science where these tools may be applied, and discuss limitations to their use and points of concern. We also discuss ethical considerations surrounding the use of LLMs to ensure that as environmental data scientists, researchers, and instructors, we can make well-considered and informed choices about engagement with these tools. Our goal is to spark forward-looking discussion and research on how as a community we can responsibly integrate generative AI technologies into our work.more » « less
-
Recognizing the challenges bilingual children face in school readiness and the potential of bilingual dialogic shared reading in improving language and literacy, this study investigates the use of a bilingual conversational agent (CA) to enhance shared reading experiences in home environments. While current CAs hold promise in fostering young children's learning, they do not typically consider the linguistic and cultural needs of bilingual children and rarely involve parents intentionally. To this end, we developed a bilingual CA, embedded within ebooks, to support children's language learning and parent engagement for Latine Spanish-English bilingual families. A week-long home-based study with 15 families indicated that the bilingual CA elicited a high level of bilingual verbal engagement from children, thereby promoting their vocabulary acquisition. It also stimulated meaningful conversations among parents and children. This study provides design implications for developing CAs for bilingual children and parents.more » « less
-
Breast cancer is the leading cancer affecting women globally. Despite deep learning models making significant strides in diagnosing and treating this disease, ensuring fair outcomes across diverse populations presents a challenge, particularly when certain demographic groups are underrepresented in training datasets. Addressing the fairness of AI models across varied demographic backgrounds is crucial. This study analyzes demographic representation within the publicly accessible Emory Breast Imaging Dataset (EMBED), which includes de-identified mammography and clinical data. We spotlight the data disparities among racial and ethnic groups and assess the biases in mammography image classification models trained on this dataset, specifically ResNet-50 and Swin Transformer V2. Our evaluation of classification accuracies across these groups reveals significant variations in model performance, highlighting concerns regarding the fairness of AI diagnostic tools. This paper emphasizes the imperative need for fairness in AI and suggests directions for future research aimed at increasing the inclusiveness and dependability of these technologies in healthcare settings. Code is available at: https://github.com/kuanhuang0624/EMBEDFairModels.more » « less
