skip to main content

Title: Development of a Lean Computational Thinking Abilities Assessment for Middle Grades Students
The recognition of middle grades as a critical juncture in CS education has led to the widespread development of CS curricula and integration efforts. The goal of many of these interventions is to develop a set of underlying abilities that has been termed computational thinking (CT). This goal presents a key challenge for assessing student learning: we must identify assessment items associated with an emergent understanding of key cognitive abilities underlying CT that avoid specialized knowledge of specific programming languages. In this work we explore the psychometric properties of assessment items appropriate for use with middle grades (US grades 6-8; ages 11-13) students. We also investigate whether these items measure a single ability dimension. Finally, we strive to recommend a "lean" set of items that can be completed in a single 50-minute class period and have high face validity. The paper makes the following contributions: 1) adds to the literature related to the emerging construct of CT, and its relationship to the existing CTt and Bebras instruments, and 2) offers a research-based CT assessment instrument for use by both researchers and educators in the field.
; ; ; ; ;
Award ID(s):
1640141 1138497
Publication Date:
Journal Name:
Proceedings of the 50th ACM Technical Symposium on Computer Science Education
Page Range or eLocation-ID:
456 - 461
Sponsoring Org:
National Science Foundation
More Like this
  1. Despite the recent proliferation of research concerning integrating computational thinking (CT) into K-5th grade curriculum, there is little literature concerning how to evaluate the quality of CT integrated curricula, especially curricula integrating CT into language arts and social studies content areas. In this paper, we present a theoretically derived rubric for the evaluation of CT integrated curricula for grades K-5 across the curriculum (math, science, language arts, social studies). Our rubric is divided into two sections. The first section provides guidelines for identifying the integration type (disciplinary, multidisciplinary, interdisciplinary, or transdisciplinary). The second section presents six categories of evaluation that further subsume nine sub-categories. The principal categories of evaluation include the following: conceptual coherence, role of computational technology, assessment, use of multiple representations, play, and equity. We include the play category as an aspect of developmental appropriateness. Play is an important pedagogical approach for learning in the early grades. Our work takes place in the context of the Computer Science (CS) for All initiative in the United States which emphasizes the goal of improving racial and gender diversity in CS participation. Therefore, creating integrated lessons that address equity is important. Our paper describes rubric development from the theoretical perspectives thatmore »underlie the inclusion of each type, category, and sub-category. Our evaluative rubric can guide future efforts to integrate CT/CS into the elementary curricula. Researchers can utilize our rubric to evaluate and analyze CT-integrated curricula, and educators can benefit from using this rubric as a guideline for curriculum development.« less
  2. Drawing, as a skill, is closely tied to many creative fields and it is a unique practice for every individual. Drawing has been shown to improve cognitive and communicative abilities, such as visual communication, problem-solving skills, students’ academic achievement, awareness of and attention to surrounding details, and sharpened analytical skills. Drawing also stimulates both sides of the brain and improves peripheral skills of writing, 3-D spatial recognition, critical thinking, and brainstorming. People are often exposed to drawing as children, drawing their families, their houses, animals, and, most notably, their imaginative ideas. These skills develop over time naturally to some extent, however, while the base concept of drawing is a basic skill, the mastery of this skill requires extensive practice and it can often be significantly impacted by the self-efficacy of an individual. Sketchtivity is an AI tool developed by Texas A&M University to facilitate the growth of drawing skills and track their performance. Sketching skill development depends in part on students’ self-efficacy associated with their drawing abilities. Gauging the drawing self-efficacy of individuals is critical in understanding the impact that this drawing practice has had with this new novel instrument, especially in contrast to traditional practicing methods. It may alsomore »be very useful for other researchers, educators, and technologists. This study reports the development and initial validation of a new 13-item measure that assesses perceived drawing self efficacy. The13 items to measure drawing self efficacy were developed based on Bandura’s guide for constructing Self-Efficacy Scales. The participants in the study consisted of 222 high school students from engineering, art, and pre-calculus classes. Internal consistency of the 13 observed items were found to be very high (Cronbach alpha: 0.943), indicating a high reliability of the scale. Exploratory Factor Analysis was performed to further investigate the variance among the 13 observed items, to find the underlying latent factors that influenced the observed items, and to see if the items needed revision. We found that a three model was the best fit for our data, given fit statistics and model interpretability. The factors are: Factor 1: Self-efficacy with respect to drawing specific objects; Factor 2: Self-efficacy with respect to drawing practically to solve problems, communicating with others, and brainstorming ideas; Factor 3: Self-efficacy with respect to drawing to create, express ideas, and use one’s imagination. An alternative four-factor model is also discussed. The purpose of our study is to inform interventions that increase self-efficacy. We believe that this assessment will be valuable especially for education researchers who implement AI-based tools to measure drawing skills.This initial validity study shows promising results for a new measure of drawing self-efficacy. Further validation with new populations and drawing classes is needed to support its use, and further psychometric testing of item-level performance. In the future, this self-efficacy assessment could be used by teachers and researchers to guide instructional interventions meant to increase drawing self-efficacy.« less
  3. The rapid expansion of K-12 CS education has made it critical to support CS teachers, many of whom are new to teaching CS, with the necessary resources and training to strengthen their understanding of CS concepts and how to effectively teach CS. CS teachers are often tasked with teaching different curricula using different programming languages in different grades or during different school years, and tend to receive different professional development (PD) for each curriculum they are required to teach. This often leads to a lack of deep understanding of the underlying CS concepts and how different curricula address the same concepts in different ways. Empowering teachers to develop a deep understanding of CS standards, and use formative assessments to recognize common student challenges associated with the standards, will enable teachers to provide more effective CS instruction, irrespective of the curriculum and/or programming language they are tasked with using. This position paper advocates supporting CS teacher professional learning by supplementing existing curriculum-specific teacher PD with standards-aligned PD that focuses on teachers' conceptual understanding of CS standards and ability to adapt instruction based on student understanding of concepts underlying the CS standards. We share concrete examples of how to design standards-aligned educativemore »resources and instructionally supportive tools that promote teachers' understanding of CS standards and common student challenges and develop teachers' formative assessment literacy, all essential components of CS pedagogical content knowledge.« less
  4. With the increasing use of online interactive environments for science and engineering education in grades K-12, there is a growing need for detailed automatic analysis of student explanations of ideas and reasoning. With the widespread adoption of the Next Generation Science Standards (NGSS), an important goal is identifying the alignment of student ideas with NGSS-defined dimensions of proficiency. We develop a set of constructed response formative assessment items that call for students to express and integrate ideas across multiple dimensions of the NGSS and explore the effectiveness of state-of-the-art neural sequence-labeling methods for identifying discourse-level expressions of ideas that align with the NGSS. We discuss challenges for idea detection task in the formative science assessment context.
  5. There is an increasing need for knowledgeable K-12 computer science (CS) teachers. It is necessary to inform teachers how to debug and help their students debug programs. Research has shown that debugging is difficult for novices because the process requires different skills from creating programs and instructing students how to debug can help them acquire these skills. To this end, we developed a CS professional development for middle grade teachers (grades 5th-8th/ages 10-13) that includes lessons on debugging. The teachers completed debugging activities that involved finding bugs in Scratch programs and explaining how they would help their students in debugging. We qualitatively analyzed their responses and found that teachers successfully identified the problem but they struggled to locate it in the code. In considering how they would help students who had such a bug, the teachers often focused on helping the student find a solution for the bug rather than on identifying the problem or its source. Finally, teachers’ ability to identify bugs and the pedagogical strategies to engage students in this process differed based on CS teaching experience and prior CS knowledge. This work contributes to our understanding of teachers’ debugging abilities and advances our knowledge on how tomore »support teachers in teaching their students how to debug their programs.« less