skip to main content


Title: Naturalistic observations of metacognition in engineering: Using observational methods to study metacognitive engagement in engineering
Abstract Background

The use of metacognition is critical to learning, especially in fields such as engineering that involve problem‐solving and difficult conceptual material. Due to limitations with current methodological approaches, new methods are needed to investigate engineering students' metacognitive engagement in learning situations that are self‐directed, such as study groups.

Purpose

Our purpose was to develop an approach to investigate the metacognitive engagement of undergraduate engineering students in self‐directed learning environments. The Naturalistic Observations of Metacognition in Engineering (NOME) Observational Protocol and Coding Strategy is a qualitative data collection method that allows researchers to observe the behaviors of students who are studying in groups to determine the student's engagement in different metacognitive practices. The NOME is intended to be used by researchers interested in studying online metacognitive behaviors without the direct interference of a methodological approach.

Design/Method

We observed three study groups where students were working on an engineering problem‐solving homework assignment. Using a taxonomic definition of metacognition, we coded episodes of observation transcripts to identify behaviors that represented key definitions in the taxonomy.

Results

We combined subcodes and descriptions of behaviors with key definitions to develop a coding strategy useful for future observational studies. Evidence of intercoder agreement and agreement in unitizing indicates that the coding strategy can reliably be used by multiple trained coders to identify metacognitive engagement.

Conclusions

The reliability evidence shows that the NOME may be a useful tool for researchers in engineering education interested in studying the metacognitive habits of engineering students in self‐directed study.

 
more » « less
NSF-PAR ID:
10124107
Author(s) / Creator(s):
 ;  
Publisher / Repository:
Wiley Blackwell (John Wiley & Sons)
Date Published:
Journal Name:
Journal of Engineering Education
Volume:
108
Issue:
4
ISSN:
1069-4730
Format(s):
Medium: X Size: p. 481-502
Size(s):
p. 481-502
Sponsoring Org:
National Science Foundation
More Like this
  1. Metacognition is the understanding of your own knowledge including what knowledge you do not have and what knowledge you do have. This includes knowledge of strategies and regulation of one’s own cognition. Studying metacognition is important because higher-order thinking is commonly used, and problem-solving skills are positively correlated with metacognition. A positive previous disposition to metacognition can improve problem-solving skills. Metacognition is a key skill in design and manufacturing, as teams of engineers must solve complex problems. Moreover, metacognition increases individual and team performance and can lead to more original ideas. This study discusses the assessment of metacognitive skills in engineering students by having the students participate in hands-on and virtual reality activities related to design and manufacturing. The study is guided by two research questions: (1) do the proposed activities affect students’ metacognition in terms of monitoring, awareness, planning, self-checking, or strategy selection, and (2) are there other components of metacognition that are affected by the design and manufacturing activities? The hypothesis is that the participation in the proposed activities will improve problem-solving skills and metacognitive awareness of the engineering students. A total of 34 undergraduate students participated in the study. Of these, 32 were male and 2 were female students. All students stated that they were interested in pursuing a career in engineering. The students were divided into two groups with the first group being the initial pilot run of the data. In this first group there were 24 students, in the second group there were 10 students. The groups’ demographics were nearly identical to each other. Analysis of the collected data indicated that problem-solving skills contribute to metacognitive skills and may develop first in students before larger metacognitive constructs of awareness, monitoring, planning, self-checking, and strategy selection. Based on this, we recommend that the problem-solving skills and expertise in solving engineering problems should be developed in students before other skills emerge or can be measured. While we are sure that the students who participated in our study have awareness as well as the other metacognitive skills in reading, writing, science, and math, they are still developing in relation to engineering problems. 
    more » « less
  2. Gardner, Stephanie (Ed.)
    Stronger metacognition, or awareness and regulation of thinking, is related to higher academic achievement. Most metacognition research has focused at the level of the individual learner. However, a few studies have shown that students working in small groups can stimulate metacognition in one another, leading to improved learning. Given the increased adoption of interactive group work in life science classrooms, there is a need to study the role of social metacognition, or the awareness and regulation of the thinking of others, in this context. Guided by the frameworks of social metacognition and evidence-based reasoning, we asked: 1) What metacognitive utterances (words, phrases, statements, or questions) do students use during small-group problem solving in an upper-division biology course? 2) Which metacognitive utterances are associated with small groups sharing higher-quality reasoning in an upper-division biology classroom? We used discourse analysis to examine transcripts from two groups of three students during breakout sessions. By coding for metacognition, we identified seven types of metacognitive utterances. By coding for reasoning, we uncovered four categories of metacognitive utterances associated with higher-quality reasoning. We offer suggestions for life science educators interested in promoting social metacognition during small-group problem solving. 
    more » « less
  3. Abstract Background

    Much of researchers’ efforts to foster wider implementation of educational innovations in STEM has focused on understanding and facilitating the implementation efforts of faculty. However, student engagement in blended learning and other innovations relies heavily on students’ self-directed learning behaviors, implying that students are likely key actors in the implementation process. This paper explores the ways in which engineering students at multiple institutions experience the self-directed selection and implementation of blended learning resources in the context of their own studies. To accomplish this, it adopts a research perspective informed by Actor-Network Theory, allowing students themselves to be perceived as individual actors and implementors rather than a population that is implemented upon.

    Results

    A thematic analysis was conducted in two parts. First, analysis identified sets of themes unique to the student experience at four participant institutions. Then, a second round of analysis identified and explored a subset of key actors represented in students’ reported experiences across all institutions. The findings show clear similarities and differences in students’ experiences of blended learning across the four institutions, with many themes echoing or building upon the results of prior research. Distinct institutional traits, the actions of the instructors, the components of the blended learning environment, and the unique needs and preferences of the students themselves all helped to shape students’ self-directed learning experiences. Students’ engagement decisions and subsequent implementations of blended learning resulted in personally appropriate, perhaps even idiosyncratic, forms of engagement with their innovative learning opportunities.

    Conclusion

    The institutional implementation of blended learning, and perhaps other educational innovations, relies in part on the self-directed decision-making of individual students. This suggests that instructors too hold an additional responsibility: to act as facilitators of their students’ implementation processes and as catalysts for growth and change in students’ learning behaviors. Developing a greater understanding of students’ implementation behaviors could inform the future implementation efforts of faculty and better empower students to succeed in the innovative classroom.

     
    more » « less
  4. Lifelong learning plays an important role in achieving success in one’s professional life. Engaging students in metacognition assists in the development of their lifelong learning abilities. Instructors can integrate reflection activities in their courses to provide multiple opportunities to students for metacognitive engagement. During reflection, students regulate their cognition by engaging themselves in three dimensions of metacognition: Planning, Monitoring, and Evaluating. Reflection is a complex process, and it takes time to reach the level of critical reflection. The purpose of the study was to investigate the change in students' level of engagement in three dimensions of metacognition when reflecting on the third and tenth-week assignments of the environmental engineering course. Data collection took place in the Fall of 2023 at a large Midwest University. Students’ responses to the assigned reflection prompts for each dimension were coded for their level of engagement in each element of the three dimensions using a revised prior coding scheme. Results showed that for both assignments, students' responses were mainly at the vague level for all elements of the three dimensions, indicating students' superficial engagement in the reflection activity. Recommendations for instructors are provided to improve students' understanding of the reflection activity and their level of engagement in the three dimensions of metacognition. 
    more » « less
  5. Self-regulation, a skillset involving taking charge of one’s own learning processes, is crucial for workplace success. Learners develop self-regulation skills through reflection where they recognize weaknesses and strengths by employing metacognitive strategies: planning, monitoring, and evaluating. Use of anchors assists learners’ engagement in reflection. The purpose of this work was to gain insight into students’ use of anchors when reflecting on their learning. The two research questions were: (1) To what extent do students link their self-evaluation and learning objective (LO) self-ratings to their reflections? and (2) What dimensions and level of metacognitive strategies do students use in their self-evaluation of and reflections on weekly problem-solving assignments? Data were upper-division engineering students’ anchors (self- evaluations, LO self-ratings) and reflection responses for one assignment. Self-evaluations and reflections were analyzed for the presence of references to LOs. The number of students who linked the anchors to their reflection were tabulated. Additionally, a revised a priori coding scheme was applied to students’ written work to determine type and level of metacognitive strategies employed. Few students linked both anchors to their reflections. Students employed low to medium levels of the metacognitive strategies in their self-evaluations and reflections, even when they linked their anchors and reflections. The evaluating strategy dominated in the self- evaluations, while planning and monitoring dominated in the reflections. Students have limited understanding of the use of anchors to guide their reflection responses. Students overall level of engagement in the metacognitive strategies indicates a need for formal instruction on reflection. 
    more » « less