skip to main content

Explore Research Products in the NSF-PAR

  • NSF Public Access
  • Search Results
  • Developing an Instrument to Measure Online Engineering Undergraduate Students’ Learning Experiences and Intentions to Persist
Title: Developing an Instrument to Measure Online Engineering Undergraduate Students’ Learning Experiences and Intentions to Persist
The purpose of this study is to develop an instrument to measure student perceptions about the learning experiences in their online undergraduate engineering courses. Online education continues to grow broadly in higher education, but the movement toward acceptance and comprehensive utilization of online learning has generally been slower in engineering. Recently, however, there have been indicators that this could be changing. For example, ABET has accredited online undergraduate engineering degrees at Stony Brook University and Arizona State University (ASU), and an increasing number of other undergraduate engineering programs also offer online courses. During this period of transition in engineering education, further investigation about the online modality in the context of engineering education is needed, and survey instrumentation can support such investigations. The instrument presented in this paper is grounded in a Model for Online Course-level Persistence in Engineering (MOCPE), which was developed by our research team by combining two motivational frameworks used to study student persistence: the Expectancy x Value Theory of Achievement Motivation (EVT), and the ARCS model of motivational design. The initial MOCPE instrument contained 79 items related to students’ perceptions about the characteristics of their courses (i.e., the online learning management system, instructor practices, and peer support), expectancies of course success, course task values, perceived course difficulties, and intention to persist in the course. Evidence of validity and reliability was collected using a three-step process. First, we tested face and content validity of the instrument with experts in online engineering education and online undergraduate engineering students. Next, the survey was administered to the online undergraduate engineering student population at a large, Southwestern public university, and an exploratory factor analysis (EFA) was conducted on the responses. Lastly, evidence of reliability was obtained by computing the internal consistency of each resulting scale. The final instrument has seven scales with 67 items across 10 factors. The Cronbach alpha values for these scales range from 0.85 to 0.97. The full paper will provide complete details about the development and psychometric evaluation of the instrument, including evidence of and reliability. The instrument described in this paper will ultimately be used as part of a larger, National Science Foundation-funded project investigating the factors influencing online undergraduate engineering student persistence. It is currently being used in the context of this project to conduct a longitudinal study intended to understand the relationships between the experiences of online undergraduate engineering students in their courses and their intentions to persist in the course. We anticipate that the instrument will be of interest and use to other engineering education researchers who are also interested in studying the population of online students.  more » « less
Award ID(s):
1825732
NSF-PAR ID:
10188464
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
Proceedings of the American Society for Engineering Education
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ( , American Society for Engineering Education Annual Conference)
    null (Ed.)
    This research paper examines the influence of interpersonal interactions on the course-level persistence intentions of online undergraduate engineering students. Online learning is increasing in enrollment and importance in engineering education. Online courses also continue to confront issues with comparatively higher course dropout levels than face-to-face courses. This study correspondingly explores relevant student perceptions of their online course experiences to better understand the factors that contribute to students’ choices to remain in or drop out of their online undergraduate engineering courses. Data presented in this study were collected during fall 2019 and spring 2020 from three ABET-accredited online undergraduate engineering courses at a large southwestern public university: electrical engineering, engineering management, and software engineering. Participants were asked to respond to surveys at 12-time points during their 7.5-week online course. Each survey measured students’ perceptions of course LMS dialog, perceptions of instructor practices, and peer support for completing the course. Participants also reported their intentions to persist in the course during each survey administration. A multi-level modeling analysis revealed that LMS dialog, perceptions of instructor practices, and peer support are related to course persistence intentions. Time was also a significant predictor of persistence intentions and indicated that the course persistence intentions decrease towards the end of the course. Additionally, interactions between demographic variables and other predictors (perceptions of course LMS dialog, perceptions of instructor practices, and perceptions of peer support) were significant. With the increase in perceptions of course LMS dialog, perceptions of instructor practices, and perceptions of peer support, there was a relatively smaller increase in the persistence intentions of veterans than non-veterans. There is relatively more increase in the persistence intentions of females than males as their perceptions of instructor practices increase. Finally, increasing perceptions of peer support led to a relatively larger increase in the persistence intentions of non-transfer students than transfer students and a relatively smaller increase in persistence intentions of students working full-time than other students. 
    more » « less
  2. ; ; ; ( , 2021 ASEE Virtual Annual Conference Content Access, Virtual Conference)
    null (Ed.)
    The purpose of this study is to re-examine the validity evidence of the engineering design self-efficacy (EDSE) scale scores by Carberry et al. (2010) within the context of secondary education. Self-efficacy refers to individuals’ belief in their capabilities to perform a domain-specific task. In engineering education, significant efforts have been made to understand the role of self-efficacy for students considering its positive impact on student outcomes such as performance and persistence. These studies have investigated and developed measures for different domains of engineering self-efficacy (e.g., general academic, domain-general, and task-specific self-efficacy). The EDSE scale is a frequently cited measure that examines task-specific self-efficacy within the domain of engineering design. The original scale contains nine items that are intended to represent the engineering design process. Initial score validity evidence was collected using a sample consisting of 202 respondents with varying degrees of engineering experience including undergraduate/graduate students and faculty members. This scale has been primarily used by researchers and practitioners with engineering undergraduate students to assess changes in their engineering design self-efficacy as a result of active learning interventions, such as project-based learning. Our work has begun to experiment using the scale in a secondary education context in conjunction with an increased introduction to engineering in K-12 education. Yet, there still is a need to examine score validity and reliability of this scale in non-undergraduate populations such as secondary school student populations. This study fills this important gap by testing construct validity of the original nine items of the EDSE scale, supporting proper use of the scale for researchers and practitioners. This study was conducted as part of a larger, e4usa project investigating the development and implementation of a yearlong project-based engineering design course for secondary school students. Evidence of construct validity and reliability was collected using a multi-step process. First, a survey that includes the EDSE scale was administered to the project participating students at nine associated secondary schools across the US at the beginning of Spring 2020. Analysis of collected data is in progress and includes Exploratory Factor Analysis (EFA) on the 137 responses. The evidence of score reliability will be obtained by computing the internal consistency of each resulting factor. The resulting factor structure and items will be analyzed by comparing it with the original EDSE scale. The full paper will provide details about the psychometric evaluation of the EDSE scale. The findings from this paper will provide insights on the future usage of the EDSE scale in the context of secondary engineering education. 
    more » « less
  3. ; ; ; ( , Research in Engineering Education Symposium & Australasian Association for Engineering Education Conference)
    Online learning is increasing in both enrollment and importance within engineering education. Online courses also continue to confront comparatively higher course dropout levels than face-to-face courses. This research paper thus aims to better understand the factors that contribute to students’ choices to remain in or drop out of their online undergraduate engineering courses. Path analysis was used to examine the impact of course perceptions and individual characteristics on students’ course-level persistence intentions. Specifically, whether students' course perceptions influenced their persistence intentions directly or indirectly, through their expectancies of course success, was tested. Data for this study were collected from three ABET-accredited online undergraduate engineering programs at a large public university in the Southwestern United States: electrical engineering, engineering management, and software engineering. A total of 138 students participated in the study during the fall 2019 (n=85) and spring 2020 (n=53) semesters. Participants responded to surveys twice weekly during their 7.5-week online course. The survey asked students about their course perceptions related to instructor practices, peer support, and course difficulty level, their expectancies in completing the course, and their course persistence intentions. This work is part of a larger National Science Foundation-funded research project dedicated to studying online student course-level persistence based on both students' self-report data and course learning management system (LMS) activity. The survey sample was consistent with reports indicating that online learners tend to be more diverse than face-to-face learners. Findings from the path analysis revealed that students' perceptions of course LMS fit, perceived course difficulty, and expectancies of course success positively and significantly predicted persistence intentions, making them the most important influences. Expectancies of course success had a direct effect on persistence intentions. The findings underscore the need to elucidate further the mechanisms through which expectancies of success influence persistence. 
    more » « less
  4. ; ; ( , ASEE Annual Conference proceedings)
    The United Nation’s Sustainable Development Goals state climate change could irreversibly affect future generations and is one of the most urgent issues facing society. To date, most education research on climate change examines middle and high school students’ knowledge without considering the link between understanding and interest to address such issues in their career. In research on students’ attitudes about sustainability, we found that half of first-year college engineering students, in our nationally representative sample of all college students at 4-year institutions (n = 937), do not believe climate change is caused by humans. This lack of belief in human-caused climate change is a significant problem in engineering education because our results also indicate engineering students who do not believe in human caused climate change are less likely to want to address climate change in their careers. This dismal finding highlights a need for improving student understanding and attitudes toward climate change in order to produce engineers prepared and interested in solving complex global problems in sustainability. To advance understanding about students’ understanding of climate change and their agency to address the issue, we developed the CLIMATE survey to measure senior undergraduate engineering students’ Climate change literacy, engineering identity, career motivations, and agency through engineering. The survey was designed for students in their final senior design, or capstone course, just prior to entering the workforce. We developed the survey using prior national surveys and newly written questions categorized into six sections: (1) career goals and motivation, (2) college experiences, (3) agency, (4) climate literacy, (5) people and the planet, and (6) demographic information. We conducted focus groups with students to establish face and content validity of the survey. We collected pilot data with 200 engineering students in upper-level engineering courses to provide validity evidence for the use of these survey items to measure students and track changes across the undergraduate curriculum for our future work. In this paper, we narrate the development of the survey supported by literature and outline the next step for further validation and distribution on a national scale. Our intent is to receive feedback and input about the questions being asked and the CLIMATE instrument. Our objective is to share the nationally representative non-identifiable responses (the estimated goal is 4,000 responses) openly with education researchers interested in students understanding about climate change, their engineering identity, career motivations, and agency through engineering. Ultimately, we want this research to become a catalyst for teaching about topics related to climate change in engineering and its implications for sustainability. 
    more » « less
  5. ; ; ( , ASEE Annual Conference proceedings)
    The United Nation’s Sustainable Development Goals state climate change could irreversibly affect future generations and is one of the most urgent issues facing society. To date, most education research on climate change examines middle and high school students’ knowledge without considering the link between understanding and interest to address such issues in their career. In research on students’ attitudes about sustainability, we found that half of first-year college engineering students, in our nationally representative sample of all college students at 4-year institutions (n = 937), do not believe climate change is caused by humans. This lack of belief in human-caused climate change is a significant problem in engineering education because our results also indicate engineering students who do not believe in human caused climate change are less likely to want to address climate change in their careers. This dismal finding highlights a need for improving student understanding and attitudes toward climate change in order to produce engineers prepared and interested in solving complex global problems in sustainability. To advance understanding about students’ understanding of climate change and their agency to address the issue, we developed the CLIMATE survey to measure senior undergraduate engineering students’ Climate change literacy, engineering identity, career motivations, and agency through engineering. The survey was designed for students in their final senior design, or capstone course, just prior to entering the workforce. We developed the survey using prior national surveys and newly written questions categorized into six sections: (1) career goals and motivation, (2) college experiences, (3) agency, (4) climate literacy, (5) people and the planet, and (6) demographic information. We conducted focus groups with students to establish face and content validity of the survey. We collected pilot data with 200 engineering students in upper-level engineering courses to provide validity evidence for the use of these survey items to measure students and track changes across the undergraduate curriculum for our future work. In this paper, we narrate the development of the survey supported by literature and outline the next step for further validation and distribution on a national scale. Our intent is to receive feedback and input about the questions being asked and the CLIMATE instrument. Our objective is to share the nationally representative non-identifiable responses (the estimated goal is 4,000 responses) openly with education researchers interested in students understanding about climate change, their engineering identity, career motivations, and agency through engineering. Ultimately, we want this research to become a catalyst for teaching about topics related to climate change in engineering and its implications for sustainability. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email