skip to main content


Title: Generalizing Predictive Models of Reading Ability in Adaptive Mathematics Software
Recent research seeks to develop more comprehensive learner models for adaptive learning software. For example, models of reading comprehension built using data from students’ use of adaptive instructional software for mathematics have recently been developed. These models aim to deliver experiences that consider factors related to learning beyond performance in the target domain for instruction. We investigate the extent to which generalization is possible for a recently developed predictive model that seeks to infer students’ reading comprehension ability (as measured by end-of-year standardized test scores) using an introductory learning experience in Carnegie Learning’s MATHia intelligent tutoring system for mathematics. Building on a model learned on data from middle school students in a single school district in a mid-western U.S. state, using that state’s end-of-year English Language Arts (ELA) standardized test score as an outcome, we consider data from a school district in a south-eastern U.S. state as well as that state’s end-of-year ELA standardized test outcome. Generalization is explored by considering prediction performance when training and testing models on data from each of the individual school districts (and for their respective state’s test outcomes) as well as pooling data from both districts together. We conclude with discussion of investigations of some algorithmic fairness characteristics of the learned models. The results suggest that a model trained on data from the smaller of the two school districts considered may achieve greater fairness in its predictions over models trained on data from the other district or both districts, despite broad, overall similarities in some demographic characteristics of the two school districts. This raises interesting questions for future research on generalizing these kinds of models as well as on ensuring algorithmic fairness of resulting models for use in real-world adaptive systems for learning.  more » « less
Award ID(s):
2000638
NSF-PAR ID:
10445554
Author(s) / Creator(s):
; ;
Editor(s):
Feng, Mingyu; Käser, Tanja; Talukdar, Partha
Date Published:
Journal Name:
Proceedings of the 16th International Conference on Educational Data Mining
Page Range / eLocation ID:
207-216
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Science in the elementary grades is often deprioritized in comparison to ELA and mathematics. We wondered, how comprehensively, frequently, and consistently is science in elementary schools’ schedules? In this study, we reviewed daily schedules for 14 schools in 9 districts across the U.S. to qualitatively examine how science is represented on the daily instructional schedule. These schools were selected as “best case scenarios” recommended by district or state science leaders as places where science is taken seriously. We complement these schedule data with data from 21 interviews with teachers, science specialists, and school leaders to better understand how science actually appears in children’s daily instructional experiences. Our findings suggest that, in these schools, science is taught comprehensively (though not as comprehensively as ELA or mathematics), has the potential for being taught frequently (even in the lower elementary grades), and is taught somewhat consistently (albeit usually in some kind of rotation with social studies). The paper closes with implications for how school schedules could be crafted to make science comprehensive, frequent, and consistent, as well as some pitfalls that could be avoided as schedules are developed. 
    more » « less
  2. Abstract

    Greenspace in schools might enhance students' academic performance. However, the literature—dominated by ecological studies at the school level in countries from the Northern Hemisphere—presents mixed evidence of a beneficial association. We evaluated the association between school greenness and student‐level academic performance in Santiago, Chile, a capital city of the Global South. This cross‐sectional study included 281,695 fourth‐grade students attending 1,498 public, charter, and private schools in Santiago city between 2014 and 2018. Student‐level academic performance was assessed using standardized test scores and indicators of attainment of learning standards in mathematics and reading. School greenness was estimated using Normalized Difference Vegetation Index (NDVI). Linear and generalized linear mixed‐effects models were fit to evaluate associations, adjusting for individual‐ and school‐level sociodemographic factors. Analyses were stratified by school type. In fully adjusted models, a 0.1 increase in school greenness was associated with higher test scores in mathematics (36.9 points, 95% CI: 2.49; 4.88) and in reading (1.84 points, 95% CI: 0.73; 2.95); as well as with higher odds of attaining learning standards in mathematics (OR: 1.20, 95% CI: 1.12; 1.28) and reading (OR: 1.07, 95% CI: 1.02; 1.13). Stratified analysis showed differences by school type, with associations of greater magnitude and strength for students attending public schools. No significant associations were detected for students in private schools. Higher school greenness was associated with improved individual‐level academic outcomes among elementary‐aged students in a capital city in South America. Our results highlight the potential of greenness in the school environment to moderate educational and environmental inequalities in urban areas.

     
    more » « less
  3. Ecosystem response to hurricane disturbance is complex and multi-faceted. The likelihood of increased frequency of severe hurricanes creates a need for the general public to understand how ecosystems respond to hurricanes. Yet, opportunities to study disturbances to complex systems are rare in U.S. K–12 schools. Educators and researchers in the Luquillo Long-Term Ecological Research program used the results of research on ecosystem response to hurricane disturbance in the Luquillo Experimental Forest as a foundation for the development of Journey to El Yunque, a web-based, middle-school curriculum unit. The program engages students in using models as evidence to develop explanations for how particular species respond to hurricane disturbance. Prior research in education has shown that engaging students in a particular role, like that of a scientist, could have detrimental effects on students’ abilities to transfer what is learned from one context to another. In this research, we sought to understand whether having students engage in authentic scientific practices could support transfer of knowledge to the abstract context of a standards-based assessment. Students were randomly assigned to engage in the program in the role of a scientist or in the role of a student learning about an ecosystem. The dependent variables included students’ comprehension of the background readings, their predictions of population changes, and their overall learning of ecology. The results indicate that taking on a scientist role during the learning activities had an indirect effect on general ecology knowledge by increasing the quality of students’ notetaking during background reading. The results also indicate that students struggled to use their knowledge to develop a robust explanation for how species respond to hurricane disturbance. Journey to El Yunque provides a framework for engaging students in authentic investigations of hurricane disturbance. Future research will examine how to improve the quality of students’ final explanations. 
    more » « less
  4. null (Ed.)
    The Statewide Coalition Supporting Underrepresented Populations in Precalculus through Organizational Redesign Toward Engineering Diversity (SC:SUPPORTED), a Design and Development Launch Pilot funded under the National Science Foundation INCLUDES program, is a coalition of secondary districts and postsecondary institutions throughout South Carolina that have joined together to address the systemic issue of mathematics preparation and placement for students pursuing or intending to pursue engineering degrees. In Year One of the project, we used individual data for all 21,656 first-year STEM-intending students enrolled in a public two- or four-year postsecondary institution with ABET-accredited engineering programs in the state to identify specific pathways with high rates of placement in or above calculus, pathways with balanced rates of placement in/below calculus, pathways with high rates of placement below calculus, and “missing” pathways: ones that produced disproportionately few engineering-intending students. From the pathways analysis we identified target locations for focus groups to identify factors that do not readily appear in institutional data, such as the impact of guidance counselor recommendations in a student’s selection of their last high school math course taken. Broad themes emerging from the focus groups provided additional insight into potential interventions at multiple points along educational pathways. These themes also contributed to both the development of a survey for statewide administration and a follow-up study to develop profiles of school district decision-making with direct and indirect effects on mathematics preparation and major selection of students from that district. As we conclude Year Two of our launch pilot, in this paper we integrate a subset of results from different aspects of the project to address both quantitative impact and qualitative context of the roles that poverty and guidance play in gaining access to engineering in South Carolina. 
    more » « less
  5. As states and districts adopt policies and programs to broaden participation in computer science, there is a growing need to effectively develop and support more computer science teachers. Instructional coaching is a form of professional development that districts can implement to provide ongoing support to teachers. In Chicago, a recent policy change made computer science a high school graduation requirement. The district adopted the Exploring Computer Science (ECS) curriculum and professional development program as the primary means for fulfilling the requirement. As part of the district’s effort to consistently offer equitable computer science experiences for all students, an emphasis was placed on supporting ECS teachers through coaching. Over the past five years, the district developed, implemented, and refined an instructional coaching model for ECS teachers. The coaching model continues to scale up across the district, with over 50 teachers participating in the 2019-2020 school year. The coaching team also adapted the model in response to the shift to remote learning during the COVID-19 pandemic. In this paper, the authors share details of the ECS coaching model in Chicago, its impact, and lessons learned from the past five years of implementation. 
    more » « less