skip to main content


Title: Assessment of Metacognitive Skills in Design and Manufacturing
Metacognition is the understanding of your own knowledge including what knowledge you do not have and what knowledge you do have. This includes knowledge of strategies and regulation of one’s own cognition. Studying metacognition is important because higher-order thinking is commonly used, and problem-solving skills are positively correlated with metacognition. A positive previous disposition to metacognition can improve problem-solving skills. Metacognition is a key skill in design and manufacturing, as teams of engineers must solve complex problems. Moreover, metacognition increases individual and team performance and can lead to more original ideas. This study discusses the assessment of metacognitive skills in engineering students by having the students participate in hands-on and virtual reality activities related to design and manufacturing. The study is guided by two research questions: (1) do the proposed activities affect students’ metacognition in terms of monitoring, awareness, planning, self-checking, or strategy selection, and (2) are there other components of metacognition that are affected by the design and manufacturing activities? The hypothesis is that the participation in the proposed activities will improve problem-solving skills and metacognitive awareness of the engineering students. A total of 34 undergraduate students participated in the study. Of these, 32 were male and 2 were female students. All students stated that they were interested in pursuing a career in engineering. The students were divided into two groups with the first group being the initial pilot run of the data. In this first group there were 24 students, in the second group there were 10 students. The groups’ demographics were nearly identical to each other. Analysis of the collected data indicated that problem-solving skills contribute to metacognitive skills and may develop first in students before larger metacognitive constructs of awareness, monitoring, planning, self-checking, and strategy selection. Based on this, we recommend that the problem-solving skills and expertise in solving engineering problems should be developed in students before other skills emerge or can be measured. While we are sure that the students who participated in our study have awareness as well as the other metacognitive skills in reading, writing, science, and math, they are still developing in relation to engineering problems.  more » « less
Award ID(s):
1830741
NSF-PAR ID:
10170318
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
ASEE Annual Conference proceedings
ISSN:
1524-4644
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Problem-solving is an iterative process that requires brainstorming, analysis of the problem, development and testing of solutions. It relies on under-standing what is known and what is unknown about the problem. That knowledge of the knowns and unknowns is called metacognition. Today’s engineers must understand their own metacognition and that of other team members to derive the best solutions for engineering problems given the different constraints. Engineers working in design and manufacturing fields confront challenges due to a lack of important metacognitive understanding of their own and their team’s problem-solving skills. This research suggests measuring metacognition within teams by using manufacturing simulations with virtual reality and eye tracking 
    more » « less
  2. This paper shares the initial findings of a three-year research project. Quantitative methods were used to develop coarse-grained understandings of undergraduate students’ self-regulation of cognition (SRC) and self-regulation of motivation (SRM) during academic problem-solving activities in two undergraduate engineering and mathematics (EM) courses. Two research questions were constructed to guide this study: (1) How are SRC and SRM strategies related to each other while solving EM problems?; and (2) How do students perceive their SRC and SRM strategies for problem-solving activities in EM courses? Two 2nd year EM courses, Engineering Statics and Ordinary Differential Equations, were purposefully selected as the contexts of the study. There were a combined total of 142 students (120 male and 20 female), across both courses, that participated in quantitative data collection using two validated surveys during spring 2022. Quantitative data were collected using two selfreport surveys: Brief Regulation of Motivation Scale (BRoMS), and the Physics Metacognitive Inventory (PMI). Although PMI was initially designed for Physics, it can be used to assess students’ metacognition for problem solving in other knowledge domains by simply revising the word “physics” to another domain knowledge. Both descriptive and inferential statistics were conducted to analyze the collected quantitative data. During data analysis we found: (1) a significant relationship between students’ strategies to selfregulate their cognition and motivation during EM problem-solving activities; (2) no significant difference between male and female’s self-regulation of cognition (SRC) and self-regulation of motivation (SRM); (3) no significant difference of SRM between students who engaged in Engineering Statics and Ordinary Differential Equation problem-solving activities; and (4) a significant difference of reported strategies in interpreting problem and evaluating strategies between those who engaged in Engineering Statics and Ordinary Differential Equation problemsolving activities. Participants reported using certain SRM strategies, such as “If I need to, I have ways of convincing myself to keep working on a tough assignment” more frequently than other strategies during problem solving. 
    more » « less
  3. Gardner, Stephanie (Ed.)
    Stronger metacognition, or awareness and regulation of thinking, is related to higher academic achievement. Most metacognition research has focused at the level of the individual learner. However, a few studies have shown that students working in small groups can stimulate metacognition in one another, leading to improved learning. Given the increased adoption of interactive group work in life science classrooms, there is a need to study the role of social metacognition, or the awareness and regulation of the thinking of others, in this context. Guided by the frameworks of social metacognition and evidence-based reasoning, we asked: 1) What metacognitive utterances (words, phrases, statements, or questions) do students use during small-group problem solving in an upper-division biology course? 2) Which metacognitive utterances are associated with small groups sharing higher-quality reasoning in an upper-division biology classroom? We used discourse analysis to examine transcripts from two groups of three students during breakout sessions. By coding for metacognition, we identified seven types of metacognitive utterances. By coding for reasoning, we uncovered four categories of metacognitive utterances associated with higher-quality reasoning. We offer suggestions for life science educators interested in promoting social metacognition during small-group problem solving. 
    more » « less
  4. This work-in-progress paper shares findings of the early stage of a 3-year research funded by the National Science Foundation. The major aim of the project is to advance engineering and mathematics (EM) education theory and practice related to students’ self-regulation of cognition and motivation skills during problem-solving activities. The self-regulation includes students’ metacognitive knowledge about task (MKT) and self-regulation of cognition (SRC). The motivational component of self-regulation (SRM) includes self-control of the motivation needed to maintain the level of engagement and deliberate practice necessary for scientific thinking and reasoning. To be effective problem-solvers, students must understand the relationship between the MKT, SRC and SRM throughout the problem-solving activities. Four research questions will guide the research: (1) How do students perceive their self-regulation of cognition (SRC) and motivation (SRM) skills for generic problem-solving activities in EM courses; (2) How does students’ metacognitive knowledge about problem-solving tasks (MKT) inform their Task interpretation?; (3) How do students’ SRC and SRM dynamically evolve?; and (4) How do students’ SRC and SRM reflect their perceptions of self-regulation of cognition and motivation for generic EM problemsolving activities? A sequential mixed-methods research design involving quantitative and qualitative methods are used to develop complementary coarse- and fine-grained understandings of undergraduate students’ SRC and SRM during academic problem-solving activities. Two 2nd year EM courses: Engineering Statics, and Ordinary Differential Equations were purposefully selected for the contexts of the study. One hundred forty two students from both courses were invited and participated in quantitative data collection using two validated surveys during spring 2022 semester. Later in the semester, qualitative data will be generated with twenty students in both courses through one-on-one interviews with students and course instructors, think-aloud protocols with students, and classroom observations. Coarse-grained understandings of students’ SRC and SRM are currently developed through analysis of quantitative data collected using self-report surveys (i.e., BRoMS and PMI). Fine-grained understandings of students’ SRC and SRM will be developed through analysis of qualitative data gathered via one-on-one interviews, think-aloud protocols, classroom observations, and course artifacts gathered as students engage in EM problem-solving activities. 
    more » « less
  5. When students repeatedly reflect, it can enhance their metacognitive abilities, including self-regulatory skills of planning, monitoring, and evaluating. In a fluid mechanics course for undergraduates at a large southeastern U.S. university, in-class problem solving in a flipped classroom was coupled with intentional metacognitive skills instruction and repeated reflection to enhance metacognition. The weekly reflective responses were coded by two analysts to identify the recurring themes and uncover evidence of the development and/or reinforcement of self-regulating behaviors for academic management. To enable a comparison, a flipped classroom without the metacognitive instruction and repeated reflection was also implemented (i.e., non-intervention group). The two cohorts completed identical final exams. Based on our preliminary analysis with year one data, a statistically and practically-significant difference between the two cohorts was found with the free-response scores on the final exam in favor of the intervention cohort that had received the metacognitive support ( p < 0.0005; Cohen's d = 0.72). Also, the Metacognitive Activities Inventory (MCAI) indicated a significantly-higher positive change in self-regulatory behavior for the intervention cohort ( p = 0.001; d = 0.50). Focus groups were conducted to gather students’ perspectives on the reflective activity, with differences found by demographic group. In addition, a significantly higher proportion of females (versus males) viewed the reflections in a positive manner ( p = 0.05). Significant associations between themes in the weekly reflections and direct knowledge measures were also uncovered. This included a positive relationship between academic self-management (i.e., diligence and carefulness) and exam performance. Overall, our preliminary results point to a desirable impact of metacognitive instruction and repeated reflection on knowledge outcomes, metacognitive skills, and self-regulatory behaviors.

     
    more » « less