skip to main content

Title: Learner Analytics in Engineering Education: A Detailed Account of Practices Used in the Cleaning and Manipulation of Learning Management System Data from Online Undergraduate Engineering Courses
This work falls under the evidence-based practice type of paper. Online undergraduate engineering education is rapidly increasing in use. The online format not only provides greater flexibility and ease of access for students, but also has lower costs for universities when compared to face-to-face courses. Even with these generally positive attributes, online courses face challenges with respect to student attrition. Numerous studies have shown that the dropout rate in online courses is higher than that for in-person courses, and topics related to online student persistence remain of interest. Data describing student interactions with their Learning Management System (LMS) provide an important lens through which online student engagement and corresponding persistence decisions can be studied, but many engineering education researchers may lack experience in working with LMS interaction data. The purpose of this paper is to provide a concrete example for other engineering education researchers of how LMS interaction data from online undergraduate engineering courses can be prepared for analysis. The work presented here is part of a larger National Science Foundation-funded study dedicated to developing a theoretical model for online undergraduate engineering student persistence based on student LMS interaction activities and patterns. Our sample dataset includes six courses, two from electrical engineering more » and four from engineering management, offered during the fall 2018 semester at a large, public southwestern university. The LMS interaction data provides details about students’ navigations to and submissions of different course elements including quizzes, assignments, discussion forums, wiki pages, attachments, modules, the syllabus, the gradebook, and course announcements. Relatedly, the features created from the data in this study can be classified into three categories: 1) learning page views, which capture student interactions with course content, 2) procedural page views, which capture student navigation to course management activities, and 3) social page views, which capture learner-to-learner and learner-to-instructor interactions. The full paper will provide the rationale and details involved in choices related to data cleaning, manipulation, and feature creation. A complete list of features will also be included. These features will ultimately be combined with associative classification to discover relationships between student-LMS interactions and persistence decisions. « less
Authors:
; ;
Award ID(s):
1825732
Publication Date:
NSF-PAR ID:
10188467
Journal Name:
Proceedings of the American Society for Engineering Education
Sponsoring Org:
National Science Foundation
More Like this
  1. This research paper examines the influence of interpersonal interactions on the course-level persistence intentions of online undergraduate engineering students. Online learning is increasing in enrollment and importance in engineering education. Online courses also continue to confront issues with comparatively higher course dropout levels than face-to-face courses. This study correspondingly explores relevant student perceptions of their online course experiences to better understand the factors that contribute to students’ choices to remain in or drop out of their online undergraduate engineering courses. Data presented in this study were collected during fall 2019 and spring 2020 from three ABET-accredited online undergraduate engineering coursesmore »at a large southwestern public university: electrical engineering, engineering management, and software engineering. Participants were asked to respond to surveys at 12-time points during their 7.5-week online course. Each survey measured students’ perceptions of course LMS dialog, perceptions of instructor practices, and peer support for completing the course. Participants also reported their intentions to persist in the course during each survey administration. A multi-level modeling analysis revealed that LMS dialog, perceptions of instructor practices, and peer support are related to course persistence intentions. Time was also a significant predictor of persistence intentions and indicated that the course persistence intentions decrease towards the end of the course. Additionally, interactions between demographic variables and other predictors (perceptions of course LMS dialog, perceptions of instructor practices, and perceptions of peer support) were significant. With the increase in perceptions of course LMS dialog, perceptions of instructor practices, and perceptions of peer support, there was a relatively smaller increase in the persistence intentions of veterans than non-veterans. There is relatively more increase in the persistence intentions of females than males as their perceptions of instructor practices increase. Finally, increasing perceptions of peer support led to a relatively larger increase in the persistence intentions of non-transfer students than transfer students and a relatively smaller increase in persistence intentions of students working full-time than other students.« less
  2. This poster will report on the research design and methodology planned for a recently funded National Science Foundation-sponsored project focused on advancing knowledge about the factors that influence the decisions of undergraduate engineering student to complete (rather than drop out of) online courses. Through the application of both social science and learner analytics-based research methods, the research will explore how students’ perceptions about the characteristics of their online undergraduate engineering courses and engagement with their course learning management system (LMS) influence their persistence. To support these studies, we draw on the undergraduate engineering student population at a large, public universitymore »in the southwestern United States that has been an early adopter of comprehensive online undergraduate engineering education. The findings from this work will be both important and timely, as the field of engineering education shows signs of embracing the online presence critical to increasing access and participation in engineering.« less
  3. This poster will report on the research design and methodology planned for a recently funded National Science Foundation-sponsored project focused on advancing knowledge about the factors that influence the decisions of undergraduate engineering student to complete (rather than drop out of) online courses. Through the application of both social science and learner analytics-based research methods, the research will explore how students’ perceptions about the characteristics of their online undergraduate engineering courses and engagement with their course learning management system (LMS) influence their persistence. To support these studies, we draw on the undergraduate engineering student population at a large, public universitymore »in the southwestern United States that has been an early adopter of comprehensive online undergraduate engineering education. The findings from this work will be both important and timely, as the field of engineering education shows signs of embracing the online presence critical to increasing access and participation in engineering.« less
  4. The purpose of this study is to develop an instrument to measure student perceptions about the learning experiences in their online undergraduate engineering courses. Online education continues to grow broadly in higher education, but the movement toward acceptance and comprehensive utilization of online learning has generally been slower in engineering. Recently, however, there have been indicators that this could be changing. For example, ABET has accredited online undergraduate engineering degrees at Stony Brook University and Arizona State University (ASU), and an increasing number of other undergraduate engineering programs also offer online courses. During this period of transition in engineering education,more »further investigation about the online modality in the context of engineering education is needed, and survey instrumentation can support such investigations. The instrument presented in this paper is grounded in a Model for Online Course-level Persistence in Engineering (MOCPE), which was developed by our research team by combining two motivational frameworks used to study student persistence: the Expectancy x Value Theory of Achievement Motivation (EVT), and the ARCS model of motivational design. The initial MOCPE instrument contained 79 items related to students’ perceptions about the characteristics of their courses (i.e., the online learning management system, instructor practices, and peer support), expectancies of course success, course task values, perceived course difficulties, and intention to persist in the course. Evidence of validity and reliability was collected using a three-step process. First, we tested face and content validity of the instrument with experts in online engineering education and online undergraduate engineering students. Next, the survey was administered to the online undergraduate engineering student population at a large, Southwestern public university, and an exploratory factor analysis (EFA) was conducted on the responses. Lastly, evidence of reliability was obtained by computing the internal consistency of each resulting scale. The final instrument has seven scales with 67 items across 10 factors. The Cronbach alpha values for these scales range from 0.85 to 0.97. The full paper will provide complete details about the development and psychometric evaluation of the instrument, including evidence of and reliability. The instrument described in this paper will ultimately be used as part of a larger, National Science Foundation-funded project investigating the factors influencing online undergraduate engineering student persistence. It is currently being used in the context of this project to conduct a longitudinal study intended to understand the relationships between the experiences of online undergraduate engineering students in their courses and their intentions to persist in the course. We anticipate that the instrument will be of interest and use to other engineering education researchers who are also interested in studying the population of online students.« less
  5. Community college engineering transfer programs prepare a significant fraction of the graduates from university engineering programs, yet face challenges from a fragmented lower division engineering core curriculum, limited scheduling options for students, and sometimes marginal enrollment patterns. In addition, most small college programs are run by one permanent faculty, making it difficult to provide lower-division engineering courses with the breadth and frequency needed for effective and timely transfer preparation. Through a grant from the National Science Foundation Improving Undergraduate STEM Education program (NSF IUSE), three community colleges from Northern California collaborated to increase the availability and accessibility of the engineeringmore »curriculum by developing resources and teaching strategies to enable small-to-medium community college engineering programs to support a comprehensive set of lower-division engineering courses. These courses can be delivered either completely online, or with limited face-to-face interactions. This paper presents the development and testing of the teaching and learning resources for an online Engineering Circuits Laboratory class, a one-unit laboratory course offered alongside the circuit theory course, which is already available in an online format. The class materials cover the use of basic instrumentation (DMM, Oscilloscope), analysis and interpretation of experimental data, circuit simulation, use of MATLAB to solve circuit equations in the real and complex domain, and exposure to the Arduino microcontroller. A systems approach to selected topics is also introduced as a way to contextualize student exposure to the material. The paper presents the results of the pilot and a second implementation of the curriculum, as well as a comparison of the outcomes of the online course with those from a regular, face-to-face course. Additionally, student surveys and interviews are used to determine student perceptions of the course resources, student use of these resources, and overall satisfaction with the course.« less