skip to main content

Explore Research Products in the NSF-PAR

  • NSF Public Access
  • Search Results
  • Learner Analytics in Engineering Education: A Detailed Account of Practices Used in the Cleaning and Manipulation of Learning Management System Data from Online Undergraduate Engineering Courses
Title: Learner Analytics in Engineering Education: A Detailed Account of Practices Used in the Cleaning and Manipulation of Learning Management System Data from Online Undergraduate Engineering Courses
This work falls under the evidence-based practice type of paper. Online undergraduate engineering education is rapidly increasing in use. The online format not only provides greater flexibility and ease of access for students, but also has lower costs for universities when compared to face-to-face courses. Even with these generally positive attributes, online courses face challenges with respect to student attrition. Numerous studies have shown that the dropout rate in online courses is higher than that for in-person courses, and topics related to online student persistence remain of interest. Data describing student interactions with their Learning Management System (LMS) provide an important lens through which online student engagement and corresponding persistence decisions can be studied, but many engineering education researchers may lack experience in working with LMS interaction data. The purpose of this paper is to provide a concrete example for other engineering education researchers of how LMS interaction data from online undergraduate engineering courses can be prepared for analysis. The work presented here is part of a larger National Science Foundation-funded study dedicated to developing a theoretical model for online undergraduate engineering student persistence based on student LMS interaction activities and patterns. Our sample dataset includes six courses, two from electrical engineering and four from engineering management, offered during the fall 2018 semester at a large, public southwestern university. The LMS interaction data provides details about students’ navigations to and submissions of different course elements including quizzes, assignments, discussion forums, wiki pages, attachments, modules, the syllabus, the gradebook, and course announcements. Relatedly, the features created from the data in this study can be classified into three categories: 1) learning page views, which capture student interactions with course content, 2) procedural page views, which capture student navigation to course management activities, and 3) social page views, which capture learner-to-learner and learner-to-instructor interactions. The full paper will provide the rationale and details involved in choices related to data cleaning, manipulation, and feature creation. A complete list of features will also be included. These features will ultimately be combined with associative classification to discover relationships between student-LMS interactions and persistence decisions.  more » « less
Award ID(s):
1825732
NSF-PAR ID:
10188467
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
Proceedings of the American Society for Engineering Education
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ( , American Society for Engineering Education Annual Conference)
    null (Ed.)
    This research paper examines the influence of interpersonal interactions on the course-level persistence intentions of online undergraduate engineering students. Online learning is increasing in enrollment and importance in engineering education. Online courses also continue to confront issues with comparatively higher course dropout levels than face-to-face courses. This study correspondingly explores relevant student perceptions of their online course experiences to better understand the factors that contribute to students’ choices to remain in or drop out of their online undergraduate engineering courses. Data presented in this study were collected during fall 2019 and spring 2020 from three ABET-accredited online undergraduate engineering courses at a large southwestern public university: electrical engineering, engineering management, and software engineering. Participants were asked to respond to surveys at 12-time points during their 7.5-week online course. Each survey measured students’ perceptions of course LMS dialog, perceptions of instructor practices, and peer support for completing the course. Participants also reported their intentions to persist in the course during each survey administration. A multi-level modeling analysis revealed that LMS dialog, perceptions of instructor practices, and peer support are related to course persistence intentions. Time was also a significant predictor of persistence intentions and indicated that the course persistence intentions decrease towards the end of the course. Additionally, interactions between demographic variables and other predictors (perceptions of course LMS dialog, perceptions of instructor practices, and perceptions of peer support) were significant. With the increase in perceptions of course LMS dialog, perceptions of instructor practices, and perceptions of peer support, there was a relatively smaller increase in the persistence intentions of veterans than non-veterans. There is relatively more increase in the persistence intentions of females than males as their perceptions of instructor practices increase. Finally, increasing perceptions of peer support led to a relatively larger increase in the persistence intentions of non-transfer students than transfer students and a relatively smaller increase in persistence intentions of students working full-time than other students. 
    more » « less
  2. ; ; ( , Proceedings of the American Society for Engineering Education)
    The purpose of this study is to develop an instrument to measure student perceptions about the learning experiences in their online undergraduate engineering courses. Online education continues to grow broadly in higher education, but the movement toward acceptance and comprehensive utilization of online learning has generally been slower in engineering. Recently, however, there have been indicators that this could be changing. For example, ABET has accredited online undergraduate engineering degrees at Stony Brook University and Arizona State University (ASU), and an increasing number of other undergraduate engineering programs also offer online courses. During this period of transition in engineering education, further investigation about the online modality in the context of engineering education is needed, and survey instrumentation can support such investigations. The instrument presented in this paper is grounded in a Model for Online Course-level Persistence in Engineering (MOCPE), which was developed by our research team by combining two motivational frameworks used to study student persistence: the Expectancy x Value Theory of Achievement Motivation (EVT), and the ARCS model of motivational design. The initial MOCPE instrument contained 79 items related to students’ perceptions about the characteristics of their courses (i.e., the online learning management system, instructor practices, and peer support), expectancies of course success, course task values, perceived course difficulties, and intention to persist in the course. Evidence of validity and reliability was collected using a three-step process. First, we tested face and content validity of the instrument with experts in online engineering education and online undergraduate engineering students. Next, the survey was administered to the online undergraduate engineering student population at a large, Southwestern public university, and an exploratory factor analysis (EFA) was conducted on the responses. Lastly, evidence of reliability was obtained by computing the internal consistency of each resulting scale. The final instrument has seven scales with 67 items across 10 factors. The Cronbach alpha values for these scales range from 0.85 to 0.97. The full paper will provide complete details about the development and psychometric evaluation of the instrument, including evidence of and reliability. The instrument described in this paper will ultimately be used as part of a larger, National Science Foundation-funded project investigating the factors influencing online undergraduate engineering student persistence. It is currently being used in the context of this project to conduct a longitudinal study intended to understand the relationships between the experiences of online undergraduate engineering students in their courses and their intentions to persist in the course. We anticipate that the instrument will be of interest and use to other engineering education researchers who are also interested in studying the population of online students. 
    more » « less
  3. ; ; ; ( , Companion Proceedings of the 9th International Conference on Learning Analytics & Knowledge)
    This poster will report on the research design and methodology planned for a recently funded National Science Foundation-sponsored project focused on advancing knowledge about the factors that influence the decisions of undergraduate engineering student to complete (rather than drop out of) online courses. Through the application of both social science and learner analytics-based research methods, the research will explore how students’ perceptions about the characteristics of their online undergraduate engineering courses and engagement with their course learning management system (LMS) influence their persistence. To support these studies, we draw on the undergraduate engineering student population at a large, public university in the southwestern United States that has been an early adopter of comprehensive online undergraduate engineering education. The findings from this work will be both important and timely, as the field of engineering education shows signs of embracing the online presence critical to increasing access and participation in engineering. 
    more » « less
  4. ; ; ; ( , International Conference on Learning Analytics & Knowledge)
    This poster will report on the research design and methodology planned for a recently funded National Science Foundation-sponsored project focused on advancing knowledge about the factors that influence the decisions of undergraduate engineering student to complete (rather than drop out of) online courses. Through the application of both social science and learner analytics-based research methods, the research will explore how students’ perceptions about the characteristics of their online undergraduate engineering courses and engagement with their course learning management system (LMS) influence their persistence. To support these studies, we draw on the undergraduate engineering student population at a large, public university in the southwestern United States that has been an early adopter of comprehensive online undergraduate engineering education. The findings from this work will be both important and timely, as the field of engineering education shows signs of embracing the online presence critical to increasing access and participation in engineering. 
    more » « less
  5. ; ; ; ( , Research in Engineering Education Symposium & Australasian Association for Engineering Education Conference)
    Online learning is increasing in both enrollment and importance within engineering education. Online courses also continue to confront comparatively higher course dropout levels than face-to-face courses. This research paper thus aims to better understand the factors that contribute to students’ choices to remain in or drop out of their online undergraduate engineering courses. Path analysis was used to examine the impact of course perceptions and individual characteristics on students’ course-level persistence intentions. Specifically, whether students' course perceptions influenced their persistence intentions directly or indirectly, through their expectancies of course success, was tested. Data for this study were collected from three ABET-accredited online undergraduate engineering programs at a large public university in the Southwestern United States: electrical engineering, engineering management, and software engineering. A total of 138 students participated in the study during the fall 2019 (n=85) and spring 2020 (n=53) semesters. Participants responded to surveys twice weekly during their 7.5-week online course. The survey asked students about their course perceptions related to instructor practices, peer support, and course difficulty level, their expectancies in completing the course, and their course persistence intentions. This work is part of a larger National Science Foundation-funded research project dedicated to studying online student course-level persistence based on both students' self-report data and course learning management system (LMS) activity. The survey sample was consistent with reports indicating that online learners tend to be more diverse than face-to-face learners. Findings from the path analysis revealed that students' perceptions of course LMS fit, perceived course difficulty, and expectancies of course success positively and significantly predicted persistence intentions, making them the most important influences. Expectancies of course success had a direct effect on persistence intentions. The findings underscore the need to elucidate further the mechanisms through which expectancies of success influence persistence. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email