More Like this

1. Can Social Robots Effectively Elicit Curiosity in STEM Topics from K-1 Students During Oral Assessments?

   https://doi.org/10.1109/EDUCON52537.2022.9766662  

   Johnson, Alexander; Martin, Alejandra; Quintero, Marlen; Bailey, Alison; Alwan, Abeer (March 2022, 2022 IEEE Global Engineering Education Conference (EDUCON))

   This paper presents the results of a pilot study that introduces social robots into kindergarten and first-grade classroom tasks. This study aims to understand 1) how effective social robots are in administering educational activities and assessments, and 2) if these interactions with social robots can serve as a gateway into learning about robotics and STEM for young children. We administered a commonly-used assessment (GFTA3) of speech production using a social robot and compared the quality of recorded responses to those obtained with a human assessor. In a comparison done between 40 children, we found no significant differences in the student responses between the two conditions over the three metrics used: word repetition accuracy, number of times additional help was needed, and similarity of prosody to the assessor. We also found that interactions with the robot were successfully able to stimulate curiosity in robotics, and therefore STEM, from a large number of the 164 student participants. 

   more » « less
2. Using Evidence from Task-Based Interviews for Development and Validation of Classroom Assessments: Approaches and Applications in Early Grades Mathematics

   https://doi.org/10.1080/10627197.2024.2398421  

   Ketterlin-Geller, Leanne R; Haider, Muhammad Qadeer; McMurrer, Jennifer (October 2024, Educational Assessment)

   This article illustrates and differentiates the unique role cognitive interviews and think-aloud interviews play in developing and validating assessments. Specifically, we describe the use of (a) cognitive interviews to gather empirical evidence to support claims about the intended construct being measured and (b) think-aloud interviews to gather evidence about the problem-solving processes students use while completing tasks assessing the intended construct. We illustrate their use in the context of a classroom assessment of an early mathematics construct – numeric relational reasoning – for kindergarten through Grade 2 students. This assessment is intended to provide teachers with data to guide their instructional decisions. We conducted 64 cognitive interviews with 32 students to collect evidence about students’ understanding of the construct. We conducted 106 think-aloud interviews with 14 students to understand how the prototypical items elicited the intended construct. The task-based interview results iteratively informed assessment development and contributed important sources of validity evidence. 

   more » « less
3. Are translated mathematics items a valid accommodation for dual language learners? Evidence from ECLS-K

   https://doi.org/10.1016/j.ecresq.2021.06.001  

   Goodrich, John Marc; Koziol, Natalie A.; Yoon, Hyeonjin (January 2021, Early childhood research quarterly)

   When measuring academic skills among students whose primary language is not English, standardized assessments are often provided in languages other than English. The degree to which alternate-language test translations yield unbiased, equitable assessment must be evaluated; however, traditional methods of investigating measurement equivalence are susceptible to confounding group differences. The primary purposes of this study were to investigate differential item functioning (DIF) and item bias across Spanish and English forms of an assessment of early mathematics skills. Secondary purposes were to investigate the presence of selection bias and demonstrate a novel approach for investigating DIF that uses a regression discontinuity design framework to control for selection bias. Data were drawn from 1,750 Spanish-speaking Kindergarteners participating in the Early Childhood Longitudinal Study, Kindergarten Class of 1998–1999, who were administered either the Spanish or English version of the mathematics assessment based on their performance on an English language screening measure. Evidence of selection bias—differences between groups in SES, age, approaches to learning, self-control, social interaction, country of birth, childcare, household composition and number in the home, books in the home, and parent involvement—highlighted limitations of a traditional approach for investigating DIF that only controlled for ability. When controlling for selection bias, only 11% of items displayed DIF, and subsequent examination of item content did not suggest item bias. Results provide evidence that the Spanish translation of the ECLS-K mathematics assessment is an equitable and unbiased assessment accommodation for young dual language learners. 

   more » « less
4. Delivering Formative Assessment and Reflecting on Student Learning as Components of Elementary Teacher Professional Learning

   Summers, Ryan; Lee, Min_Jung; Inouye, Martha; Hammack, Rebekah; Macias, Meghan (January 2025, Association for Science Teacher Education (ASTE))

   Teachers in small communities may be geographically isolated and have smaller collegial networks. Consequently, teachers in these settings may have limited exposure to contemporary strategies for engaging learners in science and engineering as suggested in the Next Generation Science Standards (NGSS). Thus, we provided a 5-day online PL experience and a year-long of modest supports (e.g., online professional learning community) to over 150 rural teachers from four states (CA, MT, ND, WY) to bridge the access gap and to enhance their instructional capabilities in teaching NGSS-aligned science and engineering lessons. Considering that the quality of the questions posed in a formative assessment impacts the quality of student thinking and what it reveals, we provided a formative assessment task, “Planning a Park” developed by Stanford NGSS Assessment Project (SNAP) and SCALE Science at WestEd, to participating teachers to implement in their classrooms. Teachers received online professional learning opportunities about the task before and after administering it in their classrooms. To understand their experiences with the task, we collected multiple data sources for triangulation, such as surveys about teachers’ preparedness to implement science lessons, teachers’ self-reported observations while delivering the task, their reflections about students’ performance, examples of student responses to the task, and interview responses from a sub-sample of teachers. As an initial analysis, we employed a descriptive coding process to capture teachers’ diverse experiences with the SCALE task (Saldaña, 2021). In this session, we will report rural teachers’ experiences with the formative assessment task that was provided as part of a year of modest supports. We believe this study will support the science education community, especially individuals preparing teachers to teach science and researchers on assessment, by sharing the benefits of implementing a formative assessment task during inservice teachers’ professional learning. 

   more » « less
5. Teachers’ interpretations of assessment results

   Folger, T.; Fornaro, C.; Bostic, J.; Sondergeld, T. (October 2021, Psychology of Mathematics Education North American)

   Olanoff, D; Johnson, K.; Spitzer, S (Ed.)

   A way to evaluate an assessment’s worth is in its contributions to student learning (Cronbach, 1988). “Classrooms are complex social environments. Economic, language, cultural, and mental health issues are just some of the key variables that need to be considered in relation to students [learning]” (Leighton, 2020, p. 27). Teachers provide a unique influence on their students’ learning through their beliefs, content knowledge, and pedagogical content knowledge (Brookhart, 2003). A classroom’s social context is an area where teacher-created assessments differ from externally-developed standardized assessments. One goal of standardized testing is to eliminate psychometric noise like social contexts by attempting to account for factors not related to the construct being measured (AERA et al., 2014). On the other hand, teacher-created assessments are contextually relevant as they are developed with certain students in mind (Brookhart, 2003). Teacher-created assessments are more likely to align with a unique social context of a classroom. The purpose of this study is to explore middle grades math teachers assessment practices and impact on student learning. 

   more » « less