An official website of the United States government
This full research paper presents the exploratory factor analysis (EFA) results for the Professional Skill Opportunities survey (PSO) we designed to measure undergraduate engineering students’ opportunities to develop and practice important nontechnical professional skills. We use Dall’alba’s “ways of being” as the theoretical framework for the survey development and generated construct definitions based on past literature, expert review, and cognitive think-aloud interviews. We administered the survey in an engineering class at the beginning of the Spring 2022 semester. After comparing the three EFA models based on goodness-of-fit indices and model interpretability aligned to the theoretical model, the researchers selected a five-factor model. The EFA result and literature on leadership and teamwork showed these two skills are highly interrelated and could be combined into one construct to stress the “sharedness” of leadership responsibilities in teams. The result allowed our team to refine our item pool, revise construct definitions, and generate new items. In future work, we will administer the revised PSO survey to the same population at the end of the same semester as further validation. We also plan to explore the relationship between professional skill development opportunities and students’ social support. We hope the PSO survey can provide educators and institutions a means to offer scaffoldings and more opportunities for professional skill development and better prepare students for the engineering workforce.
more »
« less
WIP: Think-aloud interviews for assessment of engineering students' opportunities to practice professional skills
As the need for interdisciplinary collaboration increases, industry needs engineers who are not only affluent in technical engineering skills but also efficient in skills such as communication, problem-solving, engineering ethics, and business management. As a result, engineering programs are tasked with providing students with sufficient opportunities to develop non-technical professional skills to better prepare them for the workforce. Previous research has focused on exploring how and where students tend to develop profession skills and assessments have been established to measure the level of professional skills. However, without a means to measure whether students are getting sufficient opportunities for development, it is hard for educators and engineering programs to determine whether or where scaffolding are needed. We developed an instrument to assess undergraduate engineering students’ opportunities for professional skill development. To increase content validity, we conducted 20 think-aloud interviews with students from a large Midwestern university. The aim of this WIP is two-fold. We present the preliminary results of the think-aloud interview to determine what changes need to be made to existing items and what emerging themes appear regarding to participants’ professional skill development opportunities. After thematic analysis of the interview transcripts, we revised 10 items by simplifying the grammar or altering certain words that tend to confuse participants or carry negative connotations. We found that, compared to students who have only been involved in class projects, those with co-curricular experiences tend to report more opportunities in skills related to business management principles and problem-solving skills. Co-curricular activities were also the most referenced in building communication skills. Our next step will be piloting the instrument across multiple institutions and conducting validation analysis.
more »
« less
- Award ID(s):
- 2129282
- PAR ID:
- 10343868
- Date Published:
- Journal Name:
- ASEE annual conference exposition proceedings
- ISSN:
- 2153-5868
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
The existing curriculum and models for civil engineering graduate programs assume that graduating Ph.D. students will primarily pursue career opportunities in research or academia. However, the number of civil engineering Ph.D. graduate students continues to increase, while the number of opportunities in academia for civil engineers remains stagnant. As a result, it is becoming increasingly apparent that the civil engineering graduate programs must be reevaluated to assist students entering industry after graduation. As part of a larger research study funded through the NSF Innovations in Graduate Education (IGE), we aim to answer the following research questions: 1) How can a research-to-practice model assist students in preparing for a transportation engineering career outside of academia?, 2) What impacts does the research-to-practice graduate model have on the development of transportation engineering doctoral students’ professional identity?, 3) How does the cognitive apprenticeship framework prepare doctoral students for professional practice in transportation engineering?, and 4) What influences does the research-to-practice model have on doctoral students’ motivation toward degree completion? As part of the first phase for the project, two surveys were developed: a graduate engineering student motivation survey based on Expectancy-Value-Theory, and an instrument based on the Cognitive Apprenticeship framework. The motivation survey was based on an instrument designed and validated by Brown & Matusovich (2013) which aimed to measure undergraduate engineering students' motivation towards obtaining an engineering degree. The survey prompts were reviewed and rewritten to reflect the change in context from undergraduate to graduate school. Revised survey prompts were reviewed with a group of graduate engineering students through a think aloud protocol and changes to the instrument were made to ensure consistency in interpretation of the prompts (Rodriguez-Mejia and Bodnar, 2023). The cognitive apprenticeship instrument was derived from the Maastricht Clinical Teaching Questionnaire (MCTQ), originally designed to offer clinical educators feedback on their teaching abilities, as provided by medical students during their clerkship rotations (Stalmeijer et al., 2010). To tailor it to the context of engineering graduate students, the MCTQ's 24 items were carefully examined and rephrased. A think aloud was conducted with three civil engineering graduate students to determine the effectiveness and clarity of the cognitive apprenticeship instrument. Preliminary results show that minimal clarification is needed for some items, and suggestions to include items which address support from their mentors. The other part of the project assessment involves students completing monthly reflections to obtain their opinions on specific events such as seminars or classes, and identify their perceptions of their identity as professionals, scientists, or researchers. Preliminary results suggest that the students involved place an emphasis on developing critical thinking and planning skills to become an engineering professional, but de-emphasize passion and enjoyment. This paper will report on initial findings obtained through this first phase of the IGE project.more » « less
-
CONTEXT - Judging the feasibility of solutions has become an increasingly important engineering skill as engineering problem solving has become more complex and technology-dependent. Engineering education must take care to foster engineering judgement in our students to produce robust problem solvers primed to critically evaluate and interpret output. Our work uses expertise development and dual-cognition processing theories (Dreyfus & Dreyfus, 1980; Smith, 2009; Simon, 1987) to frame such engineering judgement as engineering intuition or the ability to assess the outcome of an engineering solution and predict outcomes within an engineering scenario (Miskioğlu and Martin, 2019). PURPOSE OR GOAL - Our overarching goal is to create classroom interventions that explicitly recognize and enhance the development of engineering intuition. Accomplishing this goal requires a means of measuring engineering intuition before and after such interventions. This paper discusses our process to develop the Predicting and Evaluating Engineering Problem Solving (PEEPS) tool for measuring engineering intuition. APPROACH OR METHODOLOGY/METHODS - PEEPS is built directly on our prior qualitative work with practicing engineers, which revealed the construct of engineering intuition (Aaron et al., 2020). The emergent findings were combined with questions adapted from the Concept Assessment Tool for Statics (Steif & Dantzler, 2005) to create a preliminary survey assessing intuition. Additional items asked participants to assess their level of confidence in their answers. The survey was designed such that the statics problems could be switched out for other forms of engineering problems. Think-aloud sessions were used to check face validity and usability prior to full deployment in Spring 2021. ACTUAL OR ANTICIPATED OUTCOMES - This study details the process used to create PEEPS. Modifications were made following 19 think aloud sessions. The initial deployment in Spring 2021 resulted in 88 completed responses with responses primarily coming from white, male, aerospace engineering students who had previously performed well in their statics courses. CONCLUSIONS/RECOMMENDATIONS/SUMMARY - This work showcases a new survey designed to assess the engineering intuition of engineering students. Next steps include expanding the work to a more diverse sample of engineering students, further validity checks of the instrument, and pairing the instrument with newly created educational interventions designed to better foster engineering intuition development in students. KEYWORDS - engineering judgement, problem solving, survey developmentmore » « less
-
CONTEXT - Judging the feasibility of solutions has become an increasingly important engineering skill as engineering problem solving has become more complex and technology-dependent. Engineering education must take care to foster engineering judgement in our students to produce robust problem solvers primed to critically evaluate and interpret output. Our work uses expertise development and dual-cognition processing theories (Dreyfus & Dreyfus, 1980; Smith, 2009; Simon, 1987) to frame such engineering judgement as engineering intuition or the ability to assess the outcome of an engineering solution and predict outcomes within an engineering scenario (Miskioğlu and Martin, 2019). PURPOSE OR GOAL - Our overarching goal is to create classroom interventions that explicitly recognize and enhance the development of engineering intuition. Accomplishing this goal requires a means of measuring engineering intuition before and after such interventions. This paper discusses our process to develop the Predicting and Evaluating Engineering Problem Solving (PEEPS) tool for measuring engineering intuition. APPROACH OR METHODOLOGY/METHODS - PEEPS is built directly on our prior qualitative work with practicing engineers, which revealed the construct of engineering intuition (Aaron et al., 2020). The emergent findings were combined with questions adapted from the Concept Assessment Tool for Statics (Steif & Dantzler, 2005) to create a preliminary survey assessing intuition. Additional items asked participants to assess their level of confidence in their answers. The survey was designed such that the statics problems could be switched out for other forms of engineering problems. Think-aloud sessions were used to check face validity and usability prior to full deployment in Spring 2021. ACTUAL OR ANTICIPATED OUTCOMES - This study details the process used to create PEEPS. Modifications were made following 19 think aloud sessions. The initial deployment in Spring 2021 resulted in 88 completed responses with responses primarily coming from white, male, aerospace engineering students who had previously performed well in their statics courses. CONCLUSIONS/RECOMMENDATIONS/SUMMARY - This work showcases a new survey designed to assess the engineering intuition of engineering students. Next steps include expanding the work to a more diverse sample of engineering students, further validity checks of the instrument, and pairing the instrument with newly created educational interventions designed to better foster engineering intuition development in students. KEYWORDS - engineering judgement, problem solving, survey developmentmore » « less
-
Studies have identified gaps in the development of undergraduate students in science, technology, engineering, and mathematics (STEM). Students lack communication and problem-solving, impeding employment opportunities post-graduation. It is essential to prepare students for employment in STEM fields, as these fields remain in high demand and offer competitive wages for economic stability. Research has revealed that students gain critical thinking and problem-solving skills through students mentoring experiences. Evidence surrounding the inclusion of active learning strategies for in-classroom pedagogy has expanded in recent years, but the support mechanisms beyond the classroom remain unclear. Herein, we followed students for a decade after participation in our mentoring pre-professional training program, Nebraska STEM for You (NE STEM 4U). This phenomenological study utilized interviewing techniques and descriptive statistics to demonstrate how a midsized, metropolitan university STEM mentoring program supported the development of NE STEM 4U participants. We found that engagement in an after-school mentoring program provided participants with a model of mentorship. Participants also developed transferable professional and personal skill sets, including communication, perspectives, conflict resolution, and professional development.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
