As part of many nationwide initiatives, such as CS for All in the US [11] and Computing at School in the UK [1], many school districts all over the world have introduced computing curricula in their primary schools, expanding access to computing instruction. However, such expansion is not necessarily reaching all schools. In the United States, an early study of New York City's CS for All implementation found that schools with CS courses and activities served fewer Black and Latinx students and more White and Asian students, compared with schools without CS courses [13].
Schools can be so focused on core academic progress, particularly under-performing schools, that equitable educational opportunities for participation in content such as computer science/computational thinking (CS/CT) may be missed [8,23]. The ramifications of these missed opportunities impact long-term outcomes for college entrance, employment, and quality of life. This amplifies the need for researchers and educators to provide equitable opportunities across schools. One way to increase the likelihood for inclusion of CS/CT curricula at the primary level would be to reliably demonstrate its impact on the core academic outcomes, such as reading and math, that are most valued by schools.
The purpose of this study was to explore the relationships between computer science learning and reading and math outcomes, with the overarching goal of examining whether exposure to critical thinking and problem-solving skills through computer science learning may generalize to other content. This study was guided by the following research questions:
• How is CS/CT instruction associated with reading and math performance on standardized tests? • How do any associations between CS/CT instruction and reading and math performance apply to learners who face academic challenges?
In a quasi-experimental study of fourth-grade (ages 9-10) students, we compared the reading and math outcomes on a highstakes statewide test of those receiving CS/CT instruction with a de-identified matched sample control group who received no instruction. We found that across all students, less scaffolded, more open-ended CS/CT instruction was associated with performance gains in math, but not reading. For students who face academic challenges that influence reading and math performance [2,17,29], we found the same association with math performance gains for students with economic disadvantages and students with limited English proficiency, but not for students with disabilities. This exploratory study provides some preliminary evidence that computing supports the development of skills applicable to math, a step towards better understanding the relationship between computing and other subjects.
In the next section, we outline the theories grounding this work. In ğ3, we discuss the related work upon which we build, followed by our study design and methods in ğ4. Our results are presented in ğ5 with their discussion and implications in ğ6 and ğ7, respectively. Limitations and future work are delineated in ğ8.
This study is grounded in Neo-Piagetian theories of cognitive development more broadly, and the cognitive skills required of reading and math more specifically.
Piaget's theory posited that a child's cognition developed over time based on biological maturation and interaction with the environment [32]. Neo-Piagetian theories preserved the strengths of Piaget's theory while eliminating its weaknesses [12]. They addressed the following weaknesses of Piaget's theory: (1) it did not sufficiently explain why development between each of the stages occurs, (2) it did not adequately account for the fact that some individuals move from stage to stage faster than other individuals, and (3) its proposed universal stages of cognitive development have been empirically disproven. Two Neo-Piagetian researchers, Case and Fischer, proposed that development is not a straight progression through Piaget's main stages of development, but instead loops over all the stages, each involving their own executive control structures [6,14]. However, Fischer argued that environmental and social factors drive development, not individual factors like Case [6,14]. Based on Neo-Piagetian theories, students would build upon their existing knowledge and skills, whether from the individual or from the environment, as they learn new concepts and skills in other subjects. It is possible that students also build upon knowledge and skills acquired through CS/CT instruction when learning other subjects, such as reading and math.
To create an equitable curriculum, it is critical to understand the cognitive factors that contribute to computer science learning. When learning to program, students need to comprehend the meaning of instructions, similar to how they need to comprehend the meaning of words when reading [33]. In reading, varied text structures, conceptual density, and technical vocabulary require readers to recognize patterns, develop mental models of abstract concepts, and understand the technical meanings of disciplinary vocabulary [16]. Programming, likewise, requires comprehension of varied text structures, technical vocabulary, pattern recognition, and mental models of abstract concepts. Like reading, programs have text structures which convey meaning and have a purpose.
Although more well-studied, the relationship between mathematics and successful programming is less clear. At the university level, although math is required coursework, some researchers report that it is not often applied in computer science work. This lack of use carries into careers for software engineers [3]. At the K-12 level, educators have reported that some math concepts are too advanced and not developmentally appropriate for the age/grade levels targeted by Scratch [15]. In Scratch, programming certain kinds of movement requires the use of angles, among other math skills. According to the National Council of Teachers of Mathematics [27], understanding of angles is a standard for Grades 6-8 (ages 11-14), while Scratch is targeted for children as young as eight years old. Nonetheless, many researchers and practitioners [19,25,34,38] have found important connections between math and CS performance, though the reasons behind these relationships require further investigation.
For these reasons, research that investigates the relationships between reading, math and CS can inform curriculum development and instructional practices, and serve as motivation for educators to systematically include CS/CT as important and required curriculum.
There is plenty of work studying how reading and math contribute to computing performance. At the university level, several studies have found math skills to be factors leading to CS success [4,5,47]. As for younger learners, prior work cite English and math ability, prior computing experience, and extracurricular technology activities as contributors to success in CS learning for students ages 12-14 [19,34]. Lewis et al also found that 5th grade (ages 10-11) student performance on Scratch programming quizzes in a summer camp were highly correlated with their scores on a standardized math test [25]. A similar trend was also found in the formal school system, where learning gaps in CS/CT existed between students who were demonstrating reading and math skills below and at/above grade level [8,38].
Conversely, the reverse relationship, how computing contributes to reading and math, is less well-understood. While some argue that CS/CT and programming teaches critical thinking and problem solving beneficial to math and other subjects [31], substantiating evidence is sparse [20,30,31]. Grounded in developmental cognitive science, Pea and Kurland outline how existing evidence and assumptions fail to adequately support the idea that learning programming promotes the development of general higher mental functions [31]. They also could not find substantial evidence for the claim that programming promotes mathematical rigor, exploration, or understanding of mathematical concepts. Similarly, Palumbo cites the following reasons for the lack of evidence: lack of grounding in problem-solving theory, treatment-related issues, programming language-related issues, and the selection of an appropriate sample [30].
We build upon this body of work by more deeply investigating this reverse relationship: How does CS/CT instruction contribute to reading and math performance? This study aims to provide early evidence for CS/CT instruction as a medium to learn critical thinking and problem-solving skills that apply to other subjects and skills commonly taught at the primary level.
In this section, we describe our curriculum, study design, data sources, and analysis techniques.
Within a semester (approximately six months), students completed Scratch Act 1 [41], an introductory computational thinking (CT) curriculum modified from the Creative Computing curriculum [10], in 45-60 minute sessions every 1-2 weeks. Scratch Act 1 consists of three modules, one for each of the key CT concepts (sequence, events, and loops). Each module used Use/Modify projects to introduce the CT concept, and culminated in a Create project (see Table 1). All materials were available in both English and Spanish.
Fifteen fourth-grade (ages 9-10) teachers were recruited from a large, urban school district in Texas, USA and underwent the same professional development to teach the Scratch Act 1 curriculum. A total of 16 classrooms participated in the study, six of which were bilingual classrooms. Each classroom was assisted by an undergraduate CS researcher. Teachers were randomly assigned to either the treatment or the comparison condition, resulting in five English-only and three bilingual English and Spanish classrooms in each condition. The eight teachers in the treatment condition were taught the TIPP&SEE learning strategy, which scaffolds student exploration of example programs for Use ś> Modify activities [39]. TIPP, stands for Title, Instructions, Purpose, and Play, while SEE, stands for Sprites, Events, and Explore. Respectively, they guide students in previewing and exploring a new Scratch project. Classrooms in the comparison condition were taught Scratch Act 1 without the TIPP&SEE worksheets guiding them through the Use/Modify projects. There were a total of 92 and 101 students in the TIPP&SEE and comparison condition respectively.
An additional 162 students who did not receive CS instruction in any form were included as a de-identified control group. These students were randomly selected from the total district population of fourth graders such that the control group had a similar profile to the students who received CS instruction in terms of race/ethnicity, free and reduced lunch status (as a proxy for economic disadvantage), and special education status.
Statewide reading and math scores for 2018 and 2019 were provided by the district for each consenting student. Annually, to determine grade level readiness in reading and mathematics, Texas students complete the State of Texas Assessment of Academic Readiness (STAAR). Each STAAR question is aligned to the state curriculum standards, the Texas Essential Knowledge and Skills (TEKS), which teachers are mandated to follow as a guide to structure their lesson plans and teaching goals [42]. An external evaluation of the STAAR tests reported that development is consistent with best practices and that the Texas Education Agency (TEA) has provided evidence, including test blueprints and TEKS documentation, that support content validity [21]. This evaluation developed a predictive model to examine internal consistency reliability using Item Response Theory parameter estimates. Grade 4 projected statistics for reliability is high (0.913, Reading; 0.916, Math) and expected standard error of measurement is reasonable (2.71, Reading; 2.80, Math). At fourth-grade, the Texas Education Code [43] mandates that in mathematics, students should have the skills to use problem-solving models to support their planning, self-monitoring, and completion of work. Reading standards require development of comprehension of sequencing to carry out procedures. Standardized reading and math tests such as the STAAR assess skills that may be foundational to and associated with CS/CT.
To evaluate the relationship between CS/CT instruction and reading and math performance, we first compared across conditions: students who received instruction with the TIPP&SEE learning strategy ("TIPP&SEE students"), instruction without TIPP&SEE ("Comparison students"), and no CS/CT instruction ("No CS students"). If there were no differences between the TIPP&SEE and Comparison students, we aggregated them into one group ("CS students") to compare students who did and did not receive CS/CT instruction in any form. The different cohort configurations are summarized in Table 2. Since we tested two hypotheses on each dependent variable (reading or math score), we applied the Bonferroni correction so that our threshold for statistical significance is 𝑝 < .025, half the standard threshold of 𝑝 < .05.
No CS instruction
To study all participants as a whole, an ANCOVA was conducted on their reading and math scores to see if CS/CT instruction was associated with reading and math gains for all students. In our analysis, we controlled for 2018 scores as a covariate due to pre-test differences between the comparison and TIPP&SEE groups. We used Type III Sum of Squares for the imbalance between groups. We report 𝐹 and 𝑝 values from the ANCOVA. We also report the partial eta squared (𝜂 2 𝑝 ) effect size. The effect size indicates the magnitude of the observed effect or relationship between variables [26]. 𝜂 2 𝑝 measures the proportion of the total variance in a dependent variable (DV) that is associated with the membership of different groups defined by an independent variable (IV) [9]. For example, if an IV has a 𝜂 2 𝑝 of 0.25, that means that 25% of a DV's variance is associated with that IV. When comparing more than two groups, the Tukey post-hoc test was used to determine which groups were statistically-significantly different from one another, from which we report a 𝑝 value. improvements. Changes in reading may be associated with time, maturation, exposure to reading instruction, and potentially other factors.
However, in this study, students engaged in a less scaffolded computer science learning opportunity did demonstrate improved outcomes in math. This finding is encouraging given the importance of problem-solving and critical thinking skills for all learning. It is possible that this association is a result of exposure to and experience with the higher level math concepts embedded in the Scratch curriculum. It is also possible that the association is in response to opportunities in CS/CT curriculum to engage in higher level thinking skills with support. While more research is needed to replicate and extend this finding, generalizing of CS/CT skills to math is a very positive and desirable outcome.
It is important to note that improved outcomes in math did not result from TIPP&SEE, the more scaffolded learning opportunity. While TIPP&SEE led to better computing learning outcomes [38], it did not improve math performance. There are several possible explanations for this. First, it may be the case that the more scaffolded, less open-ended instruction resulted in less exposure to the skills that generalize to math. Second, some blocks in Scratch may expose students to more advanced math concepts, such as angles. When advanced concepts came up in Scratch, teachers in our study either explicitly taught those math concepts or worked around them [7]. It may be the case that a more structured curriculum dissuaded teachers from diverging from the curriculum cover more advanced math concepts. Further research would be necessary to investigate this trend.
For our second research question, the association between math performance gains and more open-ended, less scaffolded CS/CT instruction applied to students with economic disadvantages and with limited English proficiency, but not to students with disabilities.
A majority of students in our study (86.5%) faced some form of economic disadvantage. The selection of such a student sample was intentional, as we wanted to expand access to CS/CT instruction to students who may not have been exposed otherwise. The fact that students with economic disadvantages made up such a large proportion of our sample is a potential reason why the overall student trend applied to them. A future study with a more representative sample would be necessary to identify broader trends.
While it would be reasonable to expect that limited English proficiency would be an additional barrier to CS/CT instruction in the US, students with limited English proficiency exhibited similar patterns as the larger student body. This is likely attributable to the sophisticated and well-established bilingual English/Spanish instruction in the schools in our study and the availability of bilingual materials [36].
The reading and math performance of students with disabilities were not associated with CS/CT instruction at all. The lack of improvement in the comparison condition is predictable. Previous research in science learning has demonstrated that students with disabilities require scaffolding in order to access the curriculum [35]. Inquiry alone, as was available in the more open-ended comparison condition, is not sufficient for them. More specifically, students with learning disabilities do not succeed in open inquiry in either math or science and trends with their under-performance relative to even their lowest performing peers in reading and math [22,28].
Additionally, students with disabilities tend to have slower growth rates in their reading and math scores [17,40,46]. It may also be the case that a one-year observation of reading and math scores was not enough time to notice any effect, that one semester of CS/CT instruction was insufficient, or that the forms of CS/CT instruction offered in our study have no link to reading and math for students with disabilities. Discerning the reason for this trend difference would require further investigation. This disparity for students with disabilities suggests that while we have preliminary evidence that more open-ended CS/CT instruction is a medium for learning cognitive skills generalizable to math, it is by far not a substitute for equitable math instruction and most definitely not a panacea for addressing inequities in math, as some have claimed [31]. Further research is necessary to better understand these relationships for students with disabilities, especially since we could not disaggregate the different types of disabilities due to privacy concerns.
These findings may hopefully motivate schools to dedicate training of teachers and time toward systematic and regular inclusion of CS/CT curricula in the primary school curriculum, especially for low-performing and under-resourced districts and schools, for students of color, for multi-lingual learners, and for students with disabilities [45]. Further, the CS curriculum that was employed in this study was intentionally designed to provide equitable opportunities for all students [37,39].
Through early and successful experiences with CS/CT opportunities, the hope is that more students in poverty, multi-lingual students, students of color, and students with disabilities will enter the CS workforce, a workforce that continues to grow and is well-paid, but is historically white and male [44]. Finally, in today's world, technological knowledge and skills are important for employment, connectivity, and quality of life. School-based opportunities for technology experiences such as CS/CT provide equitable opportunities for learning that are often not available in all homes and communities [18,45]. The CS for All initiative and other movements worldwide call us to engage schools and young children in CS/CT to open the same doors for college and future employment that their more privileged peers regularly access.
This study was done in only one school district in one country and for only the statewide measure in reading and math, rather than individualized and/or diagnostic measures of reading and math performance. Further, we did not control for teacher and classroom effects due to our sample size and resource constraints. A larger scale study and replications in other school districts, other states, or even a nationwide or multinational study would be imperative to further extend and generalize this work.