skip to main content


Title: Expanding and Refining a Decision-Making Competency Inventory for Undergraduate Engineering Students
This Full Research Paper discusses ongoing work to develop a survey instrument to reliably assess undergraduate engineering student self-regulated decision-making. This work focuses on a second round of item expansion and refinement to the Decision-Making Competency Inventory (DMCI) to develop items related to learning from past decisions. The refined instrument was distributed to first-year engineering students enrolled in a large, public, land-grant institution located in the southeastern United States in the Fall of 2018. Of the approximately 1,200 students in first-year engineering courses, 883 valid surveys were randomly split into two separate samples for exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). EFA results indicated a viable four-factor solution, which was explored with the CFA. The CFA results also indicated a four-factor model was appropriate. Improving this instrument will help researchers document and understand students’ decision-making skills and how they relate to observed decisions like initial choice of major or change of major. A decision-making instrument will also be valuable in evaluating the effectiveness of interventions to help students build their decision-making competency and make adaptive choices.  more » « less
Award ID(s):
1745347
PAR ID:
10213312
Author(s) / Creator(s):
; ; ; ; ;
Date Published:
Journal Name:
2019 IEEE Frontiers in Education Conference (FIE)
Page Range / eLocation ID:
1 to 7
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. The main objective of this project is to help students learn to make decisions that lead to academic success. Our first goal is to map curriculum pathways, which begins by studying overpersistence (when a student persists in a particular major but does not make timely progress toward a degree). We seek to identify curriculum-specific indicators of overpersistence and corresponding alternative paths that could lead to success. Our second goal is to improve the structure of the Decision-Making Competency Inventory (DMCI) so that it can explain student's decision-making competency in more detail and in congruence with the Self-Regulation Model of Decision-Making. This instrument will be used to map decision-making competency to academic choices and outcomes. The third goal is to develop an Academic Dashboard as a means for sharing relevant research results with students. This will allow students to have access to the strategies, information, and stories needed to make and implement adaptive decisions. This paper highlights our progress in the fifth year of the project and our plans going forward. 
    more » « less
  2. null (Ed.)
    This complete research paper documents how confidence in choice of intended major and self-regulated decision-making competency influence whether a student changes their intended major while participating in a compulsory first-year engineering (FYE) program. Initial major, confidence in that major choice, and self-regulated decision-making competency were documented in the Fall of 2017 for students matriculating into a FYE program. Student enrollment in a major in the Fall of 2018 was connected to this data. Retention in any engineering major and in the student’s intended major were analyzed using logistic regression. 
    more » « less
  3. Ethics education has been recognized as increasingly important to engineering over the past two decades, although disagreement exists concerning how ethics can and should be taught in the classroom. With active learning strategies becoming a preferred method of instruction, a collaboration of authors from four universities (University of Pittsburgh, University of Connecticut, Rowan University and New Jersey Institute of Technology) are investigating how game-based or playful learning with strongly situated components can influence first-year engineering students’ ethical knowledge, awareness, and decision making. This paper offers an overview and results of the progress to date of this three year, NSF Improving Undergraduate STEM Education (IUSE) grant that aims to (1) characterize the ethical awareness and decision making of first-year engineering students, (2) develop game-based learning interventions focused on ethical decision making, and (3) determine how (and why) game-based approaches affect students’ ethical awareness in engineering and the advantages of such approaches over non game-based approaches. Now in its second year, the authors have conducted a preliminary analysis of first-year students' ethical knowledge and organization via a concept mapping approach and have measured students' ethical reasoning using the Defining Issues Test 2 (DIT2) and Engineering Ethics Reasoning Instrument (EERI). Further, the authors have developed a suite of ethics-driven games that have been implemented across three of the universities, engaging over 400 first-year engineering students. Evaluation data has also been gathered for further game development and to assess initial student engagement and learning. Year 1 has provided insight into where first-year engineering students “are at” in terms of ethical knowledge and reasoning when they come to college, and how game-based instruction can be effective in the development of these students into moral agents who understand the consequences of their decisions. Further results from this investigation will provide the engineering education community with a set of impactful and research-based playful learning pedagogy and assessment that will help students confront social and ethical dilemmas in their professional lives. 
    more » « less
  4. Problem. Extant measures of students’ cybersecurity self-efficacy lack sufficient evidence of validity based on internal structure. Such evidence of validity is needed to enhance confidence in conclusions drawn from use of self-efficacy measures in the cybersecurity domain. Research Question. To address this identified problem, we sought to answer our research question: What is the underlying factor structure of a new self-efficacy for Information Security measure? Method. We leveraged exploratory factor analysis (EFA) to deter- mine the number of factors underlying a new measure of student self-efficacy to conduct information security. This measure was created to align with the five elements of the information security section of the K-12 Cybersecurity Education framework. Participants were 190 undergraduate students recruited from computer science courses across the U.S. Findings. Results from the EFA indicated that a four-factor solution best fit the data while maximizing interpretability of the factors. The internal reliability of the measure was quite strong (𝛼 = .99). Implications. The psychometric quality of this measure was demonstrated, and thus evidence of validity based on internal structure has been established. Future work will conduct a confirmatory factor analysis (CFA) and assess measurement invariance across sub- groups of interest (e.g., over- vs. under-represented race/ethnicity groups, gender). 
    more » « less
  5. This project has been dedicated to advance the way computational thinking is taught to engineering undergraduate students with a multitude of social identities. It is an expectation that with the understanding of the multiple factors that affect computational thinking skills development, students succeed in enculturating to the engineering professional practice. During the third year of this project, the first major result is the conclusion of the validation process of the Engineering Computational Thinking Diagnostic (ECTD) making use of exploratory and confirmatory factor analyses (EFA-CFA). Our validation showed that the ECTD questions cluster in one factor, what we call the computational thinking factor for engineers. Other validation statistical processes (i.e. correlations, regressions, ANOVA and t-tests) proved the predictability potential use of this tool in determining how well prepared students arrive to the engineering classroom and how their prior coding experience can determine their success in introductory coding engineering courses. The second major result is the revelation that the inequities caused by the many forms of privilege that some engineering students benefit from are being exacerbated by the integration of computational thinking into introductory engineering classes. Due to pandemic-related challenges in recruiting a representative sample of participants, the majority of the self-selected participants in our research identify with groups with disproportionately large participation in engineering (specifically White and Asian) and are academically successful in engineering. To respond to this challenge we are seeking to broaden our perspective by seeking participants with failing grades for a final round of data collection, although we are well aware that students in this group are often reluctant to participate in research. The fourth and last major result is related to the position of stress versus Artificial Intelligence (AI) perceptions, both part of the ECTD instrument. The position of stress questions involved perceived difficulty and confidence level after taking the ECTD. The artificial intelligence question asked the perceived impact of AI in students’ future career prospects. Preliminary analysis is suggesting that confidence level is correlated with AI positive perceptions. Although not part of the original NSF grant, we considered AI the natural evolution of computational thinking in the formation of engineers and plan to continue our work in this direction. 
    more » « less