skip to main content

Title: Generalizing Predictive Models of Reading Ability in Adaptive Mathematics Software
Recent research seeks to develop more comprehensive learner models for adaptive learning software. For example, models of reading comprehension built using data from students’ use of adaptive instructional software for mathematics have recently been developed. These models aim to deliver experiences that consider factors related to learning beyond performance in the target domain for instruction. We investigate the extent to which generalization is possible for a recently developed predictive model that seeks to infer students’ reading comprehension ability (as measured by end-of-year standardized test scores) using an introductory learning experience in Carnegie Learning’s MATHia intelligent tutoring system for mathematics. Building on a model learned on data from middle school students in a single school district in a mid-western U.S. state, using that state’s end-of-year English Language Arts (ELA) standardized test score as an outcome, we consider data from a school district in a south-eastern U.S. state as well as that state’s end-of-year ELA standardized test outcome. Generalization is explored by considering prediction performance when training and testing models on data from each of the individual school districts (and for their respective state’s test outcomes) as well as pooling data from both districts together. We conclude with discussion of investigations of some algorithmic fairness characteristics of the learned models. The results suggest that a model trained on data from the smaller of the two school districts considered may achieve greater fairness in its predictions over models trained on data from the other district or both districts, despite broad, overall similarities in some demographic characteristics of the two school districts. This raises interesting questions for future research on generalizing these kinds of models as well as on ensuring algorithmic fairness of resulting models for use in real-world adaptive systems for learning.  more » « less
Award ID(s):
Author(s) / Creator(s):
; ;
Feng, Mingyu; Käser, Tanja; Talukdar, Partha
Date Published:
Journal Name:
Proceedings of the 16th International Conference on Educational Data Mining
Page Range / eLocation ID:
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    Greenspace in schools might enhance students' academic performance. However, the literature—dominated by ecological studies at the school level in countries from the Northern Hemisphere—presents mixed evidence of a beneficial association. We evaluated the association between school greenness and student‐level academic performance in Santiago, Chile, a capital city of the Global South. This cross‐sectional study included 281,695 fourth‐grade students attending 1,498 public, charter, and private schools in Santiago city between 2014 and 2018. Student‐level academic performance was assessed using standardized test scores and indicators of attainment of learning standards in mathematics and reading. School greenness was estimated using Normalized Difference Vegetation Index (NDVI). Linear and generalized linear mixed‐effects models were fit to evaluate associations, adjusting for individual‐ and school‐level sociodemographic factors. Analyses were stratified by school type. In fully adjusted models, a 0.1 increase in school greenness was associated with higher test scores in mathematics (36.9 points, 95% CI: 2.49; 4.88) and in reading (1.84 points, 95% CI: 0.73; 2.95); as well as with higher odds of attaining learning standards in mathematics (OR: 1.20, 95% CI: 1.12; 1.28) and reading (OR: 1.07, 95% CI: 1.02; 1.13). Stratified analysis showed differences by school type, with associations of greater magnitude and strength for students attending public schools. No significant associations were detected for students in private schools. Higher school greenness was associated with improved individual‐level academic outcomes among elementary‐aged students in a capital city in South America. Our results highlight the potential of greenness in the school environment to moderate educational and environmental inequalities in urban areas.

    more » « less
  2. As states and districts adopt policies and programs to broaden participation in computer science, there is a growing need to effectively develop and support more computer science teachers. Instructional coaching is a form of professional development that districts can implement to provide ongoing support to teachers. In Chicago, a recent policy change made computer science a high school graduation requirement. The district adopted the Exploring Computer Science (ECS) curriculum and professional development program as the primary means for fulfilling the requirement. As part of the district’s effort to consistently offer equitable computer science experiences for all students, an emphasis was placed on supporting ECS teachers through coaching. Over the past five years, the district developed, implemented, and refined an instructional coaching model for ECS teachers. The coaching model continues to scale up across the district, with over 50 teachers participating in the 2019-2020 school year. The coaching team also adapted the model in response to the shift to remote learning during the COVID-19 pandemic. In this paper, the authors share details of the ECS coaching model in Chicago, its impact, and lessons learned from the past five years of implementation. 
    more » « less
  3. Science in the elementary grades is often deprioritized in comparison to ELA and mathematics. We wondered, how comprehensively, frequently, and consistently is science in elementary schools’ schedules? In this study, we reviewed daily schedules for 14 schools in 9 districts across the U.S. to qualitatively examine how science is represented on the daily instructional schedule. These schools were selected as “best case scenarios” recommended by district or state science leaders as places where science is taken seriously. We complement these schedule data with data from 21 interviews with teachers, science specialists, and school leaders to better understand how science actually appears in children’s daily instructional experiences. Our findings suggest that, in these schools, science is taught comprehensively (though not as comprehensively as ELA or mathematics), has the potential for being taught frequently (even in the lower elementary grades), and is taught somewhat consistently (albeit usually in some kind of rotation with social studies). The paper closes with implications for how school schedules could be crafted to make science comprehensive, frequent, and consistent, as well as some pitfalls that could be avoided as schedules are developed. 
    more » « less
  4. Abstract

    Assessment is central to teaching and learning, and recently there has been a substantive shift from paper-and-pencil assessments towards technology delivered assessments such as computer-adaptive tests. Fairness is an important aspect of the assessment process, including design, administration, test-score interpretation, and data utility. The Universal Design for Learning (UDL) guidelines can inform assessment development to promote fairness; however, it is not explicitly clear how UDL and fairness may be linked through students’ conceptualizations of assessment fairness. This phenomenological study explores how middle grades students conceptualize and reason about the fairness of mathematics tests, including paper-and-pencil and technology-delivered assessments. Findings indicate that (a) students conceptualize fairness through unique notions related to educational opportunities and (b) students’ reason about fairness non-linearly. Implications of this study have potential to inform test developers and users about aspects of test fairness, as well as educators data usage from fixed-form, paper-and-pencil tests, and computer-adaptive, technology-delivered tests.

    more » « less
  5. null (Ed.)
    This paper presents an experience report from an NSF-funded researcher-practitioner partnership (RPP) project. Based on a collaboration among two public research universities and three urban school districts in the Northeast USA, the goal of the project is to establish an institutionalized middle school computer science curriculum in the districts. The CS curriculum incorporates digital literacy skills as an integral aspect of learning computer science, and is based on students developing mobile apps that provide social and community good. Here, we share our professional learning process during the project's first year, which had been developed iteratively and dynamically adjusted to a remote format in response to exigencies of Spring 2020. The paper includes analysis of three data sets from teacher-participants: (1) their questions about the nature of the project, which we categorized into three levels: project, district and teacher levels. These questions bridge the visions and knowledge among different groups of the project partners; (2) analysis of semi-structured interview conversations with more than half of the teacher-participants; and (3) teacher survey responses. Our findings include two recommendations: that RPP projects elicit teacher questions to illuminate the three levels identified, and use strategies that engage teachers in designing a professional learning process for teaching computer science. 
    more » « less