An official website of the United States government
A variety of systems have been proposed to assist users in detecting machine learning (ML) fairness issues. These systems approach bias reduction from a number of perspectives, including recommender systems, exploratory tools, and dashboards. In this paper, we seek to inform the design of these systems by examining how individuals make sense of fairness issues as they use different de-biasing affordances. In particular, we consider the tension between de-biasing recommendations which are quick but may lack nuance and ”what-if” style exploration which is time consuming but may lead to deeper understanding and transferable insights. Using logs, think-aloud data, and semi-structured interviews we find that exploratory systems promote a rich pattern of hypothesis generation and testing, while recommendations deliver quick answers which satisfy participants at the cost of reduced information exposure. We highlight design requirements and trade-offs in the design of ML fairness systems to promote accurate and explainable assessments.
more »
« less
valuating the universal screeners for number sense: A validation study. I
The Universal Screeners for Number Sense (USNS) measure the construct of number sense, which has been characterized in numerous ways over the last 50 years. A purpose of this study is to provide validity evidence regarding the USNS assessments, which may support scholarship on numeracy and practical work in K-12 schools. Findings indicate that the USNS assessments may be used confidently to measure students’ growth in number sense.
more »
« less
- Award ID(s):
- 1920621
- PAR ID:
- 10519466
- Editor(s):
- Hammock, R; Cory, B
- Publisher / Repository:
- SSMA
- Date Published:
- Volume:
- 10
- Page Range / eLocation ID:
- 6-14
- Subject(s) / Keyword(s):
- number sense, formative assessment, validity, validation
- Format(s):
- Medium: X
- Location:
- Colorado Springs, CO
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
This Brief Report presents an example of assessment validation using an argument-based approach. The instrument we developed is a Brief Assessment of Students’ Mature Number Sense, which measures a central goal in mathematics education. We chose to develop this assessment to provide an efficient way to measure the effect of instructional practices designed to improve students’ number sense. Using an argument-based framework, we first identify our proposed interpretations and uses of student scores. We then outline our argument with three claims that provide evidence connecting students’ responses on the assessment with its intended uses. Finally, we highlight why using argument-based validation benefits measure developers as well as the broader mathematics education community.more » « less
-
Students who exhibit mature number sense make sense of numbers and operations, use reasoning to notice patterns, and flexibly choose effective problem-solving strategies (McIntosh et al., 1997, https://ro.ecu.edu.au/ecuworks/6819). Due to its dispositional nature, mature number sense is typically measured through in-depth interviews or tests of strategy usage. Yet, the lack of an efficient, rigorously developed measure has made it difficult to collect systematic, replicable evidence on students’ mature number sense. To address this, we developed a brief assessment of mature number sense. The present study provides additional convergent evidence of validity for this measure with US students in grades 3-8 (8–14 years old). We compared middle school (N = 40) and upper elementary school (N = 41) scores from the brief assessment with an established, time-intensive measure (Yang, 2019, https://doi.org/10.1007/s10649-018-9874-8) and an in-depth interview of student strategy usage (Markovits & Sowder, 1994, https://doi.org/10.2307/749290). We found strong correlations (r > 0.7) across all three measures, and this held even when controlling for students’ arithmetic scores (pr > 0.6). Researchers and educators can now use the brief assessment to investigate students’ mathematical thinking and advance knowledge of a key aspect of mathematical cognition.more » « less
-
This article advances an argument-based approach to presenting validity evidence for a brief assessment of 3rd–5th grade students’ number sense. Improving students’ number sense has been a goal of K-12 mathematics education in the US since the late 1980’s, and several popular instructional practices are hypothesized to meet this need. However, there is limited evidence in the literature that elementary students’ number sense is improving. We worked to develop a brief assessment for 3rd–5th grade students and then conducted a series of studies to collect evidence that students’ scores reflect their number sense. We present the assessment here along with our intended interpretations and ways to use student scores. We then outline our validity argument, organized by four specific claims addressing the connection between student responses on the assessment and their number sense. Researchers and educators can now use the assessment to efficiently measure elementary students’ number sense.more » « less
-
Abstract Number sense, the ability to decipher quantity, forms the foundation for mathematical cognition. How number sense emerges with learning is, however, not known. Here we use a biologically-inspired neural architecture comprising cortical layers V1, V2, V3, and intraparietal sulcus (IPS) to investigate how neural representations change with numerosity training. Learning dramatically reorganized neuronal tuning properties at both the single unit and population levels, resulting in the emergence of sharply-tuned representations of numerosity in the IPS layer. Ablation analysis revealed that spontaneous number neurons observed prior to learning were not critical to formation of number representations post-learning. Crucially, multidimensional scaling of population responses revealed the emergence of absolute and relative magnitude representations of quantity, including mid-point anchoring. These learnt representations may underlie changes from logarithmic to cyclic and linear mental number lines that are characteristic of number sense development in humans. Our findings elucidate mechanisms by which learning builds novel representations supporting number sense.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript
- Conference Proceeding:
- The DOI is not currently available.
