skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


This content will become publicly available on December 1, 2026

Title: Investigating Algorithmic Bias in Affect Detectors with Constructed Categories of Student Identity
Algorithmic bias research often evaluates models in terms of traditional demographic categories (e.g., U.S. Census), but these categories may not capture nuanced, context-dependent identities relevant to learning. This study evaluates four affect detectors (boredom, confusion, engaged concentration, and frustration) developed for an adaptive math learning system. Metrics for algorithmic fairness (AUC, weighted F1, MADD) show subgroup differences across several categories that emerged from a free-response social identity survey (Twenty Statements Test; TST), including both those that mirror demographic categories (i.e., race and gender) as well as novel categories (i.e., Learner Identity, Interpersonal Style, and Sense of Competence). For demographic categories, the confusion detector performs better for boys than for girls and underperforms for West African students. Among novel categories, biases are found related to learner identity (boredom, engaged concentration, and confusion) and interpersonal style (confusion), but not for sense of competence. Results highlight the importance of using contextually grounded social identities to evaluate bias.  more » « less
Award ID(s):
2000638
PAR ID:
10658124
Author(s) / Creator(s):
; ; ; ; ; ; ; ;
Publisher / Repository:
Asia-Pacific Society for Computers in Education
Date Published:
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; (, PLOS ONE)
    Benetreau, Yann (Ed.)
    To advance understanding of doctoral student experiences and the high attrition rates among Science, Technology, Engineering, and Mathematics (STEM) doctoral students, we developed and examined the psychological profiles of different types of doctoral students. We used latent class analysis on self-reported psychological data relevant to psychological threat from 1,081 incoming doctoral students across three universities and found that the best-fitting model delineated four threat classes: Lowest Threat , Nonchalant , Engaged/Worried , and Highest Threat . These classes were associated with characteristics measured at the beginning of students’ first semester of graduate school that may influence attrition risk, including differences in academic preparation (e.g., amount of research experience), self-evaluations and perceived fit (e.g., sense of belonging), attitudes towards graduate school and academia (e.g., strength of motivation), and interpersonal relations (e.g., perceived social support). Lowest Threat students tended to report the most positive characteristics and Highest Threat students the most negative characteristics, whereas the results for Nonchalant and Engaged/Worried students were more mixed. Ultimately, we suggest that Engaged/Worried and Highest Threat students are at relatively high risk of attrition. Moreover, the demographic distributions of profiles differed, with members of groups more likely to face social identity threat (e.g., women) being overrepresented in a higher threat profile (i.e., Engaged/Worried students) and underrepresented in lower threat profiles (i.e., Lowest Threat and Nonchalant students). We conclude that doctoral students meaningfully vary in their psychological threat at the beginning of graduate study and suggest that these differences may portend divergent outcomes. 
    more » « less
  2. ; ; ; ; ; (, Journal of the Association for Information Science and Technology)
    Abstract Automated, data‐driven decision making is increasingly common in a variety of application domains. In educational software, for example, machine learning has been applied to tasks like selecting the next exercise for students to complete. Machine learning methods, however, are not always equally effective for all groups of students. Current approaches to designing fair algorithms tend to focus on statistical measures concerning a small subset of legally protected categories like race or gender. Focusing solely on legally protected categories, however, can limit our understanding of bias and unfairness by ignoring the complexities of identity. We propose an alternative approach to categorization, grounded in sociological techniques of measuring identity. By soliciting survey data and interviews from the population being studied, we can build context‐specific categories from the bottom up. The emergent categories can then be combined with extant algorithmic fairness strategies to discover which identity groups are not well‐served, and thus where algorithms should be improved or avoided altogether. We focus on educational applications but present arguments that this approach should be adopted more broadly for issues of algorithmic fairness across a variety of applications. 
    more » « less
  3. ; ; ; ; ; ; ; ; ; (, Springer Nature Switzerland)
    Adaptive learning systems are increasingly common in U.S. classrooms, but it is not yet clear whether their positive impacts are realized equally across all students. This study explores whether nuanced identity categories from open-ended self-reported data are associated with outcomes in an adaptive learning system for secondary mathematics. As a measure of impact of these social identity data, we correlate student responses for 3 categories: race and ethnicity, gender, and learning identity—a category combining student status and orientation toward learning—and total lessons completed in an adaptive learning system over one academic year. Results show the value of emergent and novel identity categories when measuring student outcomes, as learning identity was positively correlated with mathematics outcomes across two statistical tests. 
    more » « less
  4. ; ; ; ; ; ; ; ; (, 2023 IEEE Frontiers in Education Conference (FIE))
    This full paper presents the Collaborative Active Learning and Inclusiveness (CALI) inventory, and an analytical model using the CALI inventory, demographic data, mindset surveys, and knowledge mastery assessment, to explore relationships between classroom climate and student experiences. The CALI inventory enables the investigation of the impact of the student experience in an active learning classroom by distinguishing the factors that characterize the structure, social learning, and inclusive practices. The Structure Index includes components related to course setup, organization, assessment, grading, and communications. The Sociality Index includes components related to opportunities for students to interact with each other. The Inclusiveness Index includes components related to how the instructor communicates a sense of belonging to the students through a growth mindset and inclusive policies and practices. A CS Mindset Instrument was developed based on research that measured students' self-efficacy by evaluating the extent of variation in their self-perceived ability to accomplish a task, sense of belonging in computing, and professional identity development. Demographic data is collected that allows for an analysis using an intersectional lens to acknowledge the complexity of social and cultural contexts. The knowledge and mastery assessments capture changes in competency through pre-post mastery quizzes. The combination of CALI with other instruments, including those that characterize student mindset, identity, and levels of mastery, enables investigation of how various practices of inclusive and collaborative active learning have differential effects on students with different identities in computer science. 
    more » « less
  5. ; ; ; ; ; (, International Educational Data Mining Society)
    Mills, Caitlin; Alexandron, Giora; Taibi, Davide; Lo_Bosco, Giosuè; Paquette, Luc (Ed.)
    Research on epistemic emotions has often focused on how students transition between affective states (e.g., affect dynamics). More recently, studies have examined the properties of cases where a student remains in the same affective state over time, finding that the duration of a student's affective state is important for multiple learning outcomes. However, the likelihood of remaining in a given affective state has not been widely studied across different methods or systems. Additionally, the role of motivational factors in the persistence or decay of affective states remains underexplored. This study builds on two prior investigations into the exponential decay of epistemic emotions, expanding the analysis of affective chronometry by incorporating two detection methods based on student self-reports and trained observer labels in a game-based learning environment. We also examine the relationship between motivational measures and affective decay. Our findings indicate that boredom exhibits the slowest decay across both detection methods, while confusion is the least persistent. Furthermore, we found that higher situational interest and self-efficacy are associated with greater persistence in engaged concentration, as identified by both detection methods. This work provides novel insights into how motivational factors shape affective chronometry, contributing to a deeper understanding of the temporal dynamics of epistemic emotions. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email