skip to main content

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: WIP: Think-aloud interviews for assessment of engineering students' opportunities to practice professional skills
As the need for interdisciplinary collaboration increases, industry needs engineers who are not only affluent in technical engineering skills but also efficient in skills such as communication, problem-solving, engineering ethics, and business management. As a result, engineering programs are tasked with providing students with sufficient opportunities to develop non-technical professional skills to better prepare them for the workforce. Previous research has focused on exploring how and where students tend to develop profession skills and assessments have been established to measure the level of professional skills. However, without a means to measure whether students are getting sufficient opportunities for development, it is hard for educators and engineering programs to determine whether or where scaffolding are needed. We developed an instrument to assess undergraduate engineering students’ opportunities for professional skill development. To increase content validity, we conducted 20 think-aloud interviews with students from a large Midwestern university. The aim of this WIP is two-fold. We present the preliminary results of the think-aloud interview to determine what changes need to be made to existing items and what emerging themes appear regarding to participants’ professional skill development opportunities. After thematic analysis of the interview transcripts, we revised 10 items by simplifying the grammar or altering certain words that tend to confuse participants or carry negative connotations. We found that, compared to students who have only been involved in class projects, those with co-curricular experiences tend to report more opportunities in skills related to business management principles and problem-solving skills. Co-curricular activities were also the most referenced in building communication skills. Our next step will be piloting the instrument across multiple institutions and conducting validation analysis.  more » « less
Award ID(s):
2129282
PAR ID:
10343868
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
ASEE annual conference exposition proceedings
ISSN:
2153-5868
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ( , Professional skill opportunities survey: Development and exploratory factor analysis)
    This full research paper presents the exploratory factor analysis (EFA) results for the Professional Skill Opportunities survey (PSO) we designed to measure undergraduate engineering students’ opportunities to develop and practice important nontechnical professional skills. We use Dall’alba’s “ways of being” as the theoretical framework for the survey development and generated construct definitions based on past literature, expert review, and cognitive think-aloud interviews. We administered the survey in an engineering class at the beginning of the Spring 2022 semester. After comparing the three EFA models based on goodness-of-fit indices and model interpretability aligned to the theoretical model, the researchers selected a five-factor model. The EFA result and literature on leadership and teamwork showed these two skills are highly interrelated and could be combined into one construct to stress the “sharedness” of leadership responsibilities in teams. The result allowed our team to refine our item pool, revise construct definitions, and generate new items. In future work, we will administer the revised PSO survey to the same population at the end of the same semester as further validation. We also plan to explore the relationship between professional skill development opportunities and students’ social support. We hope the PSO survey can provide educators and institutions a means to offer scaffoldings and more opportunities for professional skill development and better prepare students for the engineering workforce. 
    more » « less
  2. ; ; ; ; ; ( , ASEE Conferences)
    The existing curriculum and models for civil engineering graduate programs assume that graduating Ph.D. students will primarily pursue career opportunities in research or academia. However, the number of civil engineering Ph.D. graduate students continues to increase, while the number of opportunities in academia for civil engineers remains stagnant. As a result, it is becoming increasingly apparent that the civil engineering graduate programs must be reevaluated to assist students entering industry after graduation. As part of a larger research study funded through the NSF Innovations in Graduate Education (IGE), we aim to answer the following research questions: 1) How can a research-to-practice model assist students in preparing for a transportation engineering career outside of academia?, 2) What impacts does the research-to-practice graduate model have on the development of transportation engineering doctoral students’ professional identity?, 3) How does the cognitive apprenticeship framework prepare doctoral students for professional practice in transportation engineering?, and 4) What influences does the research-to-practice model have on doctoral students’ motivation toward degree completion? As part of the first phase for the project, two surveys were developed: a graduate engineering student motivation survey based on Expectancy-Value-Theory, and an instrument based on the Cognitive Apprenticeship framework. The motivation survey was based on an instrument designed and validated by Brown & Matusovich (2013) which aimed to measure undergraduate engineering students' motivation towards obtaining an engineering degree. The survey prompts were reviewed and rewritten to reflect the change in context from undergraduate to graduate school. Revised survey prompts were reviewed with a group of graduate engineering students through a think aloud protocol and changes to the instrument were made to ensure consistency in interpretation of the prompts (Rodriguez-Mejia and Bodnar, 2023). The cognitive apprenticeship instrument was derived from the Maastricht Clinical Teaching Questionnaire (MCTQ), originally designed to offer clinical educators feedback on their teaching abilities, as provided by medical students during their clerkship rotations (Stalmeijer et al., 2010). To tailor it to the context of engineering graduate students, the MCTQ's 24 items were carefully examined and rephrased. A think aloud was conducted with three civil engineering graduate students to determine the effectiveness and clarity of the cognitive apprenticeship instrument. Preliminary results show that minimal clarification is needed for some items, and suggestions to include items which address support from their mentors. The other part of the project assessment involves students completing monthly reflections to obtain their opinions on specific events such as seminars or classes, and identify their perceptions of their identity as professionals, scientists, or researchers. Preliminary results suggest that the students involved place an emphasis on developing critical thinking and planning skills to become an engineering professional, but de-emphasize passion and enjoyment. This paper will report on initial findings obtained through this first phase of the IGE project. 
    more » « less
  3. ; ; ; ; ; ( , Research in Engineering Education Symposium & Australasian Association for Engineering Education Conference)
    CONTEXT - Judging the feasibility of solutions has become an increasingly important engineering skill as engineering problem solving has become more complex and technology-dependent. Engineering education must take care to foster engineering judgement in our students to produce robust problem solvers primed to critically evaluate and interpret output. Our work uses expertise development and dual-cognition processing theories (Dreyfus & Dreyfus, 1980; Smith, 2009; Simon, 1987) to frame such engineering judgement as engineering intuition or the ability to assess the outcome of an engineering solution and predict outcomes within an engineering scenario (Miskioğlu and Martin, 2019). PURPOSE OR GOAL - Our overarching goal is to create classroom interventions that explicitly recognize and enhance the development of engineering intuition. Accomplishing this goal requires a means of measuring engineering intuition before and after such interventions. This paper discusses our process to develop the Predicting and Evaluating Engineering Problem Solving (PEEPS) tool for measuring engineering intuition. APPROACH OR METHODOLOGY/METHODS - PEEPS is built directly on our prior qualitative work with practicing engineers, which revealed the construct of engineering intuition (Aaron et al., 2020). The emergent findings were combined with questions adapted from the Concept Assessment Tool for Statics (Steif & Dantzler, 2005) to create a preliminary survey assessing intuition. Additional items asked participants to assess their level of confidence in their answers. The survey was designed such that the statics problems could be switched out for other forms of engineering problems. Think-aloud sessions were used to check face validity and usability prior to full deployment in Spring 2021. ACTUAL OR ANTICIPATED OUTCOMES - This study details the process used to create PEEPS. Modifications were made following 19 think aloud sessions. The initial deployment in Spring 2021 resulted in 88 completed responses with responses primarily coming from white, male, aerospace engineering students who had previously performed well in their statics courses. CONCLUSIONS/RECOMMENDATIONS/SUMMARY - This work showcases a new survey designed to assess the engineering intuition of engineering students. Next steps include expanding the work to a more diverse sample of engineering students, further validity checks of the instrument, and pairing the instrument with newly created educational interventions designed to better foster engineering intuition development in students. KEYWORDS - engineering judgement, problem solving, survey development 
    more » « less
  4. ; ; ; ; ; ( , Research in Engineering Education Symposium & Australasian Association for Engineering Education Conference)
    CONTEXT - Judging the feasibility of solutions has become an increasingly important engineering skill as engineering problem solving has become more complex and technology-dependent. Engineering education must take care to foster engineering judgement in our students to produce robust problem solvers primed to critically evaluate and interpret output. Our work uses expertise development and dual-cognition processing theories (Dreyfus & Dreyfus, 1980; Smith, 2009; Simon, 1987) to frame such engineering judgement as engineering intuition or the ability to assess the outcome of an engineering solution and predict outcomes within an engineering scenario (Miskioğlu and Martin, 2019). PURPOSE OR GOAL - Our overarching goal is to create classroom interventions that explicitly recognize and enhance the development of engineering intuition. Accomplishing this goal requires a means of measuring engineering intuition before and after such interventions. This paper discusses our process to develop the Predicting and Evaluating Engineering Problem Solving (PEEPS) tool for measuring engineering intuition. APPROACH OR METHODOLOGY/METHODS - PEEPS is built directly on our prior qualitative work with practicing engineers, which revealed the construct of engineering intuition (Aaron et al., 2020). The emergent findings were combined with questions adapted from the Concept Assessment Tool for Statics (Steif & Dantzler, 2005) to create a preliminary survey assessing intuition. Additional items asked participants to assess their level of confidence in their answers. The survey was designed such that the statics problems could be switched out for other forms of engineering problems. Think-aloud sessions were used to check face validity and usability prior to full deployment in Spring 2021. ACTUAL OR ANTICIPATED OUTCOMES - This study details the process used to create PEEPS. Modifications were made following 19 think aloud sessions. The initial deployment in Spring 2021 resulted in 88 completed responses with responses primarily coming from white, male, aerospace engineering students who had previously performed well in their statics courses. CONCLUSIONS/RECOMMENDATIONS/SUMMARY - This work showcases a new survey designed to assess the engineering intuition of engineering students. Next steps include expanding the work to a more diverse sample of engineering students, further validity checks of the instrument, and pairing the instrument with newly created educational interventions designed to better foster engineering intuition development in students. KEYWORDS - engineering judgement, problem solving, survey development 
    more » « less
  5. ; ; ; ; ; ( , ASEE Annual Conference & Exposition, Baltimore, MD)
    This work-in-progress (WIP) paper aims to elucidate how students have developed professional skills since the onset of the COVID-19 pandemic and who are the people who have provided skill development opportunities. Because of the way social distancing affected engineering education during the pandemic, developing professional skills may have been a challenge for engineering students. While online courses and virtual meetings allowed students to remain in contact with faculty and each other, the opportunities to continue having deep relationships (i.e., strong ties) were sparse. Our paper presents an early look at findings from the qualitative phase of an explanatory mixed methods study conducted with 1,234 undergraduates from 13 schools in the US. Our ongoing thematic qualitative analysis reveals that the changes that accompanied social distancing and periods of emergency remote teaching caused by COVID-19 have reinforced different opportunities to develop professional skills than prior to the pandemic. While some students expressed they had fewer opportunities to develop professional skills, participants also identified opportunities to (1) hone written communication skills when inperson discussions were reduced and (2) leverage knowledge from family members to continue developing professionally. Our next steps include finishing the qualitative analysis phase of the project and mixing the qualitative and quantitative data to develop overarching findings that the engineering education community can use to understand how students’ professional skills develop and how to promote that development even during times of educational disruption. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email