skip to main content


The NSF Public Access Repository (NSF-PAR) system and access will be unavailable from 11:00 PM ET on Thursday, May 23 until 2:00 AM ET on Friday, May 24 due to maintenance. We apologize for the inconvenience.

Title: Students’ neurocognition changes during engineering design when thinking aloud
The purpose of this study was to measure the neurocognitive effects of think aloud when engineering students were designing. Thinking aloud is a commonly applied protocol in engineering design education research. The process involves students verbalizing what they are thinking as they perform a task. Students are asked to say what comes into their mind. This often includes what they are looking at, thinking, doing, and feeling. It provides insight into the student’s mental state and their cognitive processes when developing design ideas. Think aloud provides a richer understanding about how, what and why students’ design compared to solely evaluating their final product or performance. The results show that Ericsson and Simon's claim that there is no interference due to think-aloud is not supported by this study and more research is required to untangle the effect of think-aloud.  more » « less
Award ID(s):
Author(s) / Creator(s):
; ; ; ; ;
Date Published:
Journal Name:
NSF EEC Grantees Conference 2022
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    The think-aloud protocol provides researchers an insight into the designer's mental state, but little is understood about how thinking aloud influences design. The study presented in this paper sets out to measure the cognitive and neurocognitive changes in designers when thinking aloud. Engineering students (n=50) were randomly assigned to the think-aloud or control group. Students were outfitted with a functional near-infrared spectroscopy band. Students were asked to design a personal entertainment system. The think-aloud group spent significantly less time designing. Their design sketches included significantly fewer words. The think-aloud group also required significantly more resources in the left and right dorsolateral prefrontal cortex (DLPFC). The left DLPFC is often recruited for language processing, and the right DLPFC is involved in visual representation and problem-solving. The faster depletion of neurocognitive resources may have contributed to less time designing. Thinking aloud influences design cognition and neurocognition, but these effects are only now becoming apparent. More research and the adoption of neuroscience techniques can help shed light on these differences.

    more » « less
  2. Engineers are called to play an important role in addressing the complex problems of our global society, such as climate change and global health care. In order to adequately address these complex problems, engineers must be able to identify and incorporate into their decision making relevant aspects of systems in which their work is contextualized, a skill often referred to as systems thinking. However, within engineering, research on systems thinking tends to emphasize the ability to recognize potentially relevant constituent elements and parts of an engineering problem, rather than how these constituent elements and parts are embedded in broader economic, sociocultural, and temporal contexts and how all of these must inform decision making about problems and solutions. Additionally, some elements of systems thinking, such as an awareness of a particular sociocultural context or the coordination of work among members of a cross-disciplinary team, are not always recognized as core engineering skills, which alienates those whose strengths and passions are related to, for example, engineering systems that consider and impact social change. Studies show that women and minorities, groups underrepresented within engineering, are drawn to engineering in part for its potential to address important social issues. Emphasizing the importance of systems thinking and developing a more comprehensive definition of systems thinking that includes both constituent parts and contextual elements of a system will help students recognize the relevance and value of these other elements of engineering work and support full participation in engineering by a diverse group of students. We provide an overview of our study, in which we are examining systems thinking across a range of expertise to develop a scenario-based assessment tool that educators and researchers can use to evaluate engineering students’ systems thinking competence. Consistent with the aforementioned need to define and study systems thinking in a comprehensive, inclusive manner, we begin with a definition of systems thinking as a holistic approach to problem solving in which linkages and interactions of the immediate work with constituent parts, the larger sociocultural context, and potential impacts over time are identified and incorporated into decision making. In our study, we seek to address two key questions: 1) How do engineers of different levels of education and experience approach problems that require systems thinking? and 2) How do different types of life, educational, and work experiences relate to individuals’ demonstrated level of expertise in solving systems thinking problems? Our study is comprised of three phases. The first two phases include a semi-structured interview with engineering students and professionals about their experiences solving a problem requiring systems thinking and a think-aloud interview in which participants are asked to talk through how they would approach a given engineering scenario and later reflect on the experiences that inform their thinking. Data from these two phases will be used to develop a written assessment tool, which we will test by administering the written instrument to undergraduate and graduate engineering students in our third study phase. Our paper describes our study design and framing and includes preliminary findings from the first phase of our study. 
    more » « less
  3. Abstract

    Critical thinking, which can be defined as the evidence‐based ways in which people decide what to trust and what to do, is an important competency included in many undergraduate science, technology, engineering, and mathematics (STEM) courses. To help instructors effectively measure critical thinking, we developed the Biology Lab Inventory of Critical Thinking in Ecology (Eco‐BLIC), a freely available, closed‐response assessment of undergraduate students' critical thinking in ecology. The Eco‐BLIC includes ecology‐based experimental scenarios followed by questions that measure how students decide on what to trust and what to do next. Here, we present the development of the Eco‐BLIC using tests of validity and reliability. Using student responses to questions and think‐aloud interviews, we demonstrate the effectiveness of the Eco‐BLIC at measuring students' critical thinking skills. We find that while students generally think like experts while evaluating what to trust, students' responses are less expert‐like when deciding on what to do next.

    more » « less
  4. Abstract Background

    Engineering students inconsistently apply equilibrium when solving problems in statics, but few studies have explored why. Visual cognition studies suggest that features of the visual representations we use to teach students influence what domain knowledge they use to solve problems. However, few studies have explored how visual representations influence what problem‐solving strategies and domain knowledge students of different levels of expertise use when solving problems that require them to create and coordinate multiple representations.


    This study addressed the following research question: How do students with different levels of expertise coordinate their problem‐solving strategies, problem‐solving heuristics, and representation features when sketching their shear force and bending moment diagrams?


    We conducted think‐aloud interviews while students sketched shear force and bending moment diagrams. These interviews were subsequently analyzed using the constant comparative method to examine the effect of representations on students' problem‐solving approaches.


    Three themes emerged from the data: Students used heuristics that are based on perceptually salient features to sketch their shear force and bending moment diagrams; students across levels of expertise rely on theobject translationheuristic rather than equilibrium problem‐solving schema to sketch and reason through their shear force and bending moment diagrams, and domain knowledge aids students' ability to resolve conflicting heuristics. Our findings suggest that students primarily rely on heuristics triggered by representation features they notice.


    Students engaged with shear force and bending moment diagrams not as a way to describe systems that are not accelerating but as a series of representations that “should go to zero” or arrows that make things “not zero.”

    more » « less
  5. Abstract

    The research presented in this paper investigated the changes that occur in the prefrontal cortex (PFC) when new ideas are introduced during engineering design. Undergraduate and graduate engineering students (n = 25) were outfitted with a functional near-infrared spectroscopy (fNIRS) headband. Students were asked to design a personal entertainment system while thinking aloud. New ideas were timestamped with the fNIRS data across 48 channels grouped into eight regions within the PFC. The data were preprocessed using temporal derivative distribution repair motion correction, finite impulse response bandpass filter, and the modified beer-lambert law to convert optical density into hemoglobin concentration. Baseline neurocognitive activation and physiological noise were removed. The study found a significant decrease in oxygenated hemoglobin in the left dorsolateral prefrontal cortex and a subregion of the left ventrolateral prefrontal cortex when new ideas were introduced during design. This finding begins to provide a neurocognitive signature of what a new idea looks like as it arises in the brain. This could be used to develop tools and techniques to inhibit this brain region or use this insight to predict when designers will experience a new idea based on their neural activation.

    more » « less