More Like this

1. Can Social Robots Effectively Elicit Curiosity in STEM Topics from K-1 Students During Oral Assessments?

   https://doi.org/10.1109/EDUCON52537.2022.9766662  

   Johnson, Alexander; Martin, Alejandra; Quintero, Marlen; Bailey, Alison; Alwan, Abeer (March 2022, 2022 IEEE Global Engineering Education Conference (EDUCON))

   This paper presents the results of a pilot study that introduces social robots into kindergarten and first-grade classroom tasks. This study aims to understand 1) how effective social robots are in administering educational activities and assessments, and 2) if these interactions with social robots can serve as a gateway into learning about robotics and STEM for young children. We administered a commonly-used assessment (GFTA3) of speech production using a social robot and compared the quality of recorded responses to those obtained with a human assessor. In a comparison done between 40 children, we found no significant differences in the student responses between the two conditions over the three metrics used: word repetition accuracy, number of times additional help was needed, and similarity of prosody to the assessor. We also found that interactions with the robot were successfully able to stimulate curiosity in robotics, and therefore STEM, from a large number of the 164 student participants. 

   more » « less
2. Using Evidence from Task-Based Interviews for Development and Validation of Classroom Assessments: Approaches and Applications in Early Grades Mathematics

   https://doi.org/10.1080/10627197.2024.2398421  

   Ketterlin-Geller, Leanne R; Haider, Muhammad Qadeer; McMurrer, Jennifer (October 2024, Educational Assessment)

   This article illustrates and differentiates the unique role cognitive interviews and think-aloud interviews play in developing and validating assessments. Specifically, we describe the use of (a) cognitive interviews to gather empirical evidence to support claims about the intended construct being measured and (b) think-aloud interviews to gather evidence about the problem-solving processes students use while completing tasks assessing the intended construct. We illustrate their use in the context of a classroom assessment of an early mathematics construct – numeric relational reasoning – for kindergarten through Grade 2 students. This assessment is intended to provide teachers with data to guide their instructional decisions. We conducted 64 cognitive interviews with 32 students to collect evidence about students’ understanding of the construct. We conducted 106 think-aloud interviews with 14 students to understand how the prototypical items elicited the intended construct. The task-based interview results iteratively informed assessment development and contributed important sources of validity evidence. 

   more » « less
3. Are translated mathematics items a valid accommodation for dual language learners? Evidence from ECLS-K

   https://doi.org/10.1016/j.ecresq.2021.06.001  

   Goodrich, John Marc; Koziol, Natalie A.; Yoon, Hyeonjin (January 2021, Early childhood research quarterly)

   When measuring academic skills among students whose primary language is not English, standardized assessments are often provided in languages other than English. The degree to which alternate-language test translations yield unbiased, equitable assessment must be evaluated; however, traditional methods of investigating measurement equivalence are susceptible to confounding group differences. The primary purposes of this study were to investigate differential item functioning (DIF) and item bias across Spanish and English forms of an assessment of early mathematics skills. Secondary purposes were to investigate the presence of selection bias and demonstrate a novel approach for investigating DIF that uses a regression discontinuity design framework to control for selection bias. Data were drawn from 1,750 Spanish-speaking Kindergarteners participating in the Early Childhood Longitudinal Study, Kindergarten Class of 1998–1999, who were administered either the Spanish or English version of the mathematics assessment based on their performance on an English language screening measure. Evidence of selection bias—differences between groups in SES, age, approaches to learning, self-control, social interaction, country of birth, childcare, household composition and number in the home, books in the home, and parent involvement—highlighted limitations of a traditional approach for investigating DIF that only controlled for ability. When controlling for selection bias, only 11% of items displayed DIF, and subsequent examination of item content did not suggest item bias. Results provide evidence that the Spanish translation of the ECLS-K mathematics assessment is an equitable and unbiased assessment accommodation for young dual language learners. 

   more » « less
4. Teachers’ interpretations of assessment results

   Folger, T.; Fornaro, C.; Bostic, J.; Sondergeld, T. (October 2021, Psychology of Mathematics Education North American)

   Olanoff, D; Johnson, K.; Spitzer, S (Ed.)

   A way to evaluate an assessment’s worth is in its contributions to student learning (Cronbach, 1988). “Classrooms are complex social environments. Economic, language, cultural, and mental health issues are just some of the key variables that need to be considered in relation to students [learning]” (Leighton, 2020, p. 27). Teachers provide a unique influence on their students’ learning through their beliefs, content knowledge, and pedagogical content knowledge (Brookhart, 2003). A classroom’s social context is an area where teacher-created assessments differ from externally-developed standardized assessments. One goal of standardized testing is to eliminate psychometric noise like social contexts by attempting to account for factors not related to the construct being measured (AERA et al., 2014). On the other hand, teacher-created assessments are contextually relevant as they are developed with certain students in mind (Brookhart, 2003). Teacher-created assessments are more likely to align with a unique social context of a classroom. The purpose of this study is to explore middle grades math teachers assessment practices and impact on student learning. 

   more » « less
5. Exploring the Relationships Among Middle School Students’ Peer Interactions, Task Efficiency, and Learning Engagement in Game-Based Learning

   https://doi.org/10.1177/1046878120907940  

   Moon, Jewoong; Ke, Fengfeng (March 2020, Simulation & Gaming)

   Background. Middle school students’ math anxiety and low engagement have been major issues in math education. In order to reduce their anxiety and support their math learning, game-based learning (GBL) has been implemented. GBL research has underscored the role of social dynamics to facilitate a qualitative understanding of students’ knowledge. Whereas students’ peer interactions have been deemed a social dynamic, the relationships among peer interaction, task efficiency, and learning engagement were not well understood in previous empirical studies. Method. This mixed-method research implemented E-Rebuild, which is a 3D architecture game designed to promote students’ math problem-solving skills. We collected a total of 102 50-minutes gameplay sessions performed by 32 middle school students. Using video-captured and screen-recorded gameplaying sessions, we implemented behavior observations to measure students’ peer interaction efficiency, task efficiency, and learning engagement. We used association analyses, sequential analysis, and thematic analysis to explain how peer interaction promoted students’ task efficiency and learning engagement. Results. Students’ peer interactions were negatively related to task efficiency and learning engagement. There were also different gameplay patterns by students’ learning/task-relevant peer-interaction efficiency (PIE) level. Students in the low PIE group tended to progress through game tasks more efficiently than those in the high PIE group. The results of qualitative thematic analysis suggested that the students in the low PIE group showed more reflections on game-based mathematical problem solving, whereas those with high PIE experienced distractions during gameplay. Discussion. This study confirmed that students’ peer interactions without purposeful and knowledge-constructive collaborations led to their low task efficiency, as well as low learning engagement. The study finding shows further design implications: (1) providing in-game prompts to stimulate students’ math-related discussions and (2) developing collaboration contexts that legitimize students’ interpersonal knowledge exchanges with peers. 

   more » « less