skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Gathering response process data for a problem-solving measure through whole-class think alouds.
Response process validity evidence provides a window into a respondent’s cognitive processing. The purpose of this study is to describe a new data collection tool called a whole-class think aloud (WCTA). This work is performed as part of test development for a series of problem-solving measures to be used in elementary and middle grades. Data from third-grade students were collected in a 1-1 think aloud setting and compared to data from similar students as part of WCTAs. Findings indicated that students performed similarly on the items when the two think-aloud settings were compared. Respondents also needed less encouragement to share ideas aloud during the WCTA compared to the 1-1 think aloud. They also communicated feeling more comfortable in the WCTA setting compared to the 1-1 think aloud. Drawing the findings together, WCTAs functioned as well if not better, than 1-1 think alouds for the purpose of contextualizing third grade students’ cognitive processes. Future studies using WCTAs are recommended to explore their limitations and other factors that might impact their success as data gathering tools.  more » « less
Award ID(s):
1720646 1720661
PAR ID:
10252192
Author(s) / Creator(s):
Date Published:
Journal Name:
Applied measurement in education
Volume:
34
Issue:
1
ISSN:
0895-7347
Page Range / eLocation ID:
46-60
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; (, NSF EEC Grantees Conference 2022)
    ASEE (Ed.)
    The purpose of this study was to measure the neurocognitive effects of think aloud when engineering students were designing. Thinking aloud is a commonly applied protocol in engineering design education research. The process involves students verbalizing what they are thinking as they perform a task. Students are asked to say what comes into their mind. This often includes what they are looking at, thinking, doing, and feeling. It provides insight into the student’s mental state and their cognitive processes when developing design ideas. Think aloud provides a richer understanding about how, what and why students’ design compared to solely evaluating their final product or performance. The results show that Ericsson and Simon's claim that there is no interference due to think-aloud is not supported by this study and more research is required to untangle the effect of think-aloud. 
    more » « less
  2. ; ; (, Educational Assessment)
    This article illustrates and differentiates the unique role cognitive interviews and think-aloud interviews play in developing and validating assessments. Specifically, we describe the use of (a) cognitive interviews to gather empirical evidence to support claims about the intended construct being measured and (b) think-aloud interviews to gather evidence about the problem-solving processes students use while completing tasks assessing the intended construct. We illustrate their use in the context of a classroom assessment of an early mathematics construct – numeric relational reasoning – for kindergarten through Grade 2 students. This assessment is intended to provide teachers with data to guide their instructional decisions. We conducted 64 cognitive interviews with 32 students to collect evidence about students’ understanding of the construct. We conducted 106 think-aloud interviews with 14 students to understand how the prototypical items elicited the intended construct. The task-based interview results iteratively informed assessment development and contributed important sources of validity evidence. 
    more » « less
  3. (, RESPECT 2020 Digital Proceedings)
    null (Ed.)
    We describe a research study aimed at understanding the basic code reasoning challenges of elementary school children. The research targets third to fifth grade African American students as a step towards making computer science accessible to all children. The study was conducted in a summer camp with 40 students and replicated with 20 new students the following summer. Also participating in the second summer camp were 19 returning students. For data collection, the study uses code-tracing activities involving concepts such as variables, assignments, operators, and sequencing. Performance data is automatically collected in the background as children engage in the activities incorporated in a video game and also through think-aloud sessions. Results include common code understanding challenges for all children. 
    more » « less
  4. ; ; ; (, Proceedings of the Design Society)
    Abstract The think-aloud protocol provides researchers an insight into the designer's mental state, but little is understood about how thinking aloud influences design. The study presented in this paper sets out to measure the cognitive and neurocognitive changes in designers when thinking aloud. Engineering students (n=50) were randomly assigned to the think-aloud or control group. Students were outfitted with a functional near-infrared spectroscopy band. Students were asked to design a personal entertainment system. The think-aloud group spent significantly less time designing. Their design sketches included significantly fewer words. The think-aloud group also required significantly more resources in the left and right dorsolateral prefrontal cortex (DLPFC). The left DLPFC is often recruited for language processing, and the right DLPFC is involved in visual representation and problem-solving. The faster depletion of neurocognitive resources may have contributed to less time designing. Thinking aloud influences design cognition and neurocognition, but these effects are only now becoming apparent. More research and the adoption of neuroscience techniques can help shed light on these differences. 
    more » « less
  5. ; ; (, roceedings of the forty-fourth annual meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education)
    Lischka, A. E.; Dyer, E. B.; Jones, R. S.; Lovett, J. N.; Strayer, J.; Drown, S. (Ed.)
    Utilizing an innovative and theoretically-grounded approach, we extend the work of cognitive scientists and mathematics educators who have previously documented the impact of comparison on students’ learning in algebra with the goal of transforming the learning that occurs in eighth- grade geometry classrooms. The purpose of this paper is to examine the types of comparisons participants made during think aloud interviews when engaging with curricular materials that have them examine multiple solution strategies. This research seeks to extend the work of using comparisons in algebra to determine if using comparisons in geometry will help improve students’ mathematical understanding. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email