skip to main content


Title: Demographic Mediation of the Relationship Between Engagement and Performance in a Blended Dynamics Engineering Course
ABSTRACT CONTEXT This paper examines an engineering dynamics course at Purdue University that was specifically designed to create an active, blended, and collaborative environment. In addition to in-person classes and support, students have access to blended content such as solution videos, mechanics visualizations, a course discussion forum, and interactive simulations. PURPOSE Many studies have shown that students’ engagement in an online discussion forum enhances their learning performance (Davies & Graff, 2005; Hrastinski, 2008). However, our previous research showed that students’ engagement in the online forum of our dynamics course differed significantly across students’ demographics. We showed that women, white, or Asian American students were more likely to be involved in online discussions than men, international, or Hispanic students (Duan et al., 2018). In this paper, we take the previous analysis further by examining whether the observed differences in online student engagement mediate or moderate student performance. APPROACH To answer our research question, we will first investigate the mediation effect by creating two models. A first model with race/international status as the mediating variable and gender identity as a control variable, and a second model with gender identity as the mediating variable and race/international status as a control. Second, we will investigate the moderation effect of demographic factors by creating a regression model including interaction terms to show the relationship of each demographic’s discussion forum engagement to overall performance. The goal of investigating these interaction terms is to determine if a moderating relationship exists where demographic factors impact online engagement, which in turn impact course performance. CONCLUSIONS We find that gender identity is the only significant demographic factor that moderates the effect of a student’s engagement on their performance. Based on the findings of our previous work, students of various racial and ethnic identities do engage differently in the discussion forum. However, this analysis was unable to detect any significant difference in student engagement based on demographics. Our paper contributes to understanding the mechanisms through which students’ engagement can translate into academic performance by focusing on their demographic background. The moderating role of students’ demographic background calls for a more targeted design of instructional tools in blended and collaborative environments to better support students from various demographic backgrounds.  more » « less
Award ID(s):
1915574
NSF-PAR ID:
10355868
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
Proceedings of REES AAEE 2021
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. High levels of stress and anxiety are common amongst college students, particularly engineering students. Students report lack of sleep, grades, competition, change in lifestyle, and other significant stressors throughout their undergraduate education (1, 2). Stress and anxiety have been shown to negatively impact student experience (3-6), academic performance (6-8), and retention (9). Previous studies have focused on identifying factors that cause individual students stress while completing undergraduate engineering degree programs (1). However, it not well-understood how a culture of stress is perceived and is propagated in engineering programs or how this culture impacts student levels of identification with engineering. Further, the impact of student stress has not been directly considered in engineering regarding recruitment, retention, and success. Therefore, our guiding research question is: Does the engineering culture create stress for students that hinder their engineering identity development? To answer our research question, we designed a sequential mixed methods study with equal priority of quantitative survey data and qualitative individual interviews. Our study participants are undergraduate engineering students across all levels and majors at a large, public university. Our sample goal is 2000 engineering student respondents. We combined three published surveys to build our quantitative data collection instrument, including the Depression Anxiety Stress Scales (DASS), Identification with engineering subscale, and Engineering Department Inclusion Level subscale. The objective of the quantitative instrument is to illuminate individual perceptions of the existence of an engineering stress culture (ESC) and create an efficient tool to measure the impact ESC on engineering identity development. Specifically, we seek to understand the relationships among the following constructs; 1) identification with engineering, 2) stress and anxiety, and 3) feelings of inclusion within their department. The focus of this paper presents the results of the pilot of the proposed instrument with 20 participants and a detailed data collection and analysis process. In an effort to validate our instrument, we conducted a pilot study to refine our data collection process and the results will guide the data collection for the larger study. In addition to identifying relationships among construct, the survey data will be further analyzed to specify which demographics are mediating or moderating factors of these relationships. For example, does a student’s 1st generation status influence their perception of stress or engineering identity development? Our analysis may identify discipline-specific stressors and characterize culture components that promote student anxiety and stress. Our objective is to validate our survey instrument and use it to inform the protocol for the follow-up interviews to gain a deeper understanding of the responses to the survey instrument. Understanding what students view as stressful and how students identify stress as an element of program culture will support the development of interventions to mitigate student stress. References 1. Schneider L (2007) Perceived stress among engineering students. A Paper Presented at St. Lawrence Section Conference. Toronto, Canada. Retrieved from: www. asee. morrisville. edu. 2. Ross SE, Niebling BC, & Heckert TM (1999) Sources of stress among college students. Social psychology 61(5):841-846. 3. Goldman CS & Wong EH (1997) Stress and the college student. Education 117(4):604-611. 4. Hudd SS, et al. (2000) Stress at college: Effects on health habits, health status and self-esteem. College Student Journal 34(2):217-228. 5. Macgeorge EL, Samter W, & Gillihan SJ (2005) Academic Stress, Supportive Communication, and Health A version of this paper was presented at the 2005 International Communication Association convention in New York City. Communication Education 54(4):365-372. 6. Burt KB & Paysnick AA (2014) Identity, stress, and behavioral and emotional problems in undergraduates: Evidence for interaction effects. Journal of college student development 55(4):368-384. 7. Felsten G & Wilcox K (1992) Influences of stress and situation-specific mastery beliefs and satisfaction with social support on well-being and academic performance. Psychological Reports 70(1):291-303. 8. Pritchard ME & Wilson GS (2003) Using emotional and social factors to predict student success. Journal of college student development 44(1):18-28. 9. Zhang Z & RiCharde RS (1998) Prediction and Analysis of Freshman Retention. AIR 1998 Annual Forum Paper. 
    more » « less
  2. This study investigated patterns in the development of computational thinking practices in the context of the Exploring Computer Science (ECS) program, a high school introductory CS course and professional development program designed to foster deep engagement through equitable inquiry around CS concepts. Past research indicates that the personal relevance of the ECS experience influences students' expectancy-value towards computer science. Expectancy-value is a construct that is predictive of career choices. We extended our research to examine whether expectancy-value influences the development of computational thinking practices. This study took place in the context of two ECS implementation projects across two states. Twenty teachers, who implemented ECS in 2016–17, participated in the research. There were 906 students who completed beginning and end of year surveys and assessments. The surveys included demographic questions, a validated expectancy-value scale, and questions about students' course experiences. The assessments were developed and validated by SRI International as a companion to the ECS course. Overall, student performance statistically increased from pretest to posttest with effect size of 0.74. There were no statistically significant differences in performance by gender or race/ethnicity. These results are consistent with earlier findings that a personally relevant course experience positively influences students' expectancy for success. These results expanded on prior research by indicating that students' expectancy-value for computer science positively influenced student learning. 
    more » « less
  3. null (Ed.)
    Over the past two decades, educators have used computer-supported collaborative learning (CSCL) to integrate technology with pedagogy to improve student engagement and learning outcomes. Researchers have also explored the diverse affordances of CSCL, its contributions to engineering instruction, and its effectiveness in K-12 STEM education. However, the question of how students use CSCL resources in undergraduate engineering classrooms remains largely unexplored. This study examines the affordances of a CSCL environment utilized in a sophomore dynamics course with particular attention given to the undergraduate engineering students’ use of various CSCL resources. The resources include a course lecturebook, instructor office hours, a teaching assistant help room, online discussion board, peer collaboration, and demonstration videos. This qualitative study uses semi-structured interview data collected from nine mechanical engineering students (four women and five men) who were enrolled in a dynamics course at a large public research university in Eastern Canada. The interviews focused on the individual student’s perceptions of the school, faculty, students, engineering courses, and implemented CSCL learning environment. The thematic analysis was conducted to analyze the transcribed interviews using a qualitative data analysis software (Nvivo). The analysis followed a six step process: (1) reading interview transcripts multiple times and preliminary in vivo codes; (2) conducting open coding by coding interesting or salient features of the data; (3) collecting codes and searching for themes; (4) reviewing themes and creating a thematic map; (5) finalizing themes and their definitions; and (6) compiling findings. This study found that the students’ use of CSCL resources varied depending on the students’ personal preferences, as well as their perceptions of the given resource’s value and its potential to enhance their learning. For example, the dynamics lecturebook, which had been redesigned to encourage problem solving and note-taking, fostered student collaborative problem solving with their peers. In contrast, the professor’s example video solutions had much more of an influence on students’ independent problem-solving processes. The least frequently used resource was the course’s online discussion forum, which could be used as a means of communication. The findings reveal how computer-supported collaborative learning (CSCL) environments enable engineering students to engage in multiple learning opportunities with diverse and flexible resources to both address and to clarify their personal learning needs. This study strongly recommends engineering instructors adapt a CSCL environment for implementation in their own unique classroom context. 
    more » « less
  4. Many undergraduate students encounter struggle as they navigate academic, financial, and social contexts of higher education. The transition to emergency online instruction during the Spring of 2020 due to the COVID-19 pandemic exacerbated these struggles. To assess college students’ struggles during the transition to online learning in undergraduate biology courses, we surveyed a diverse collection of students ( n = 238) at an R2 research institution in the Southeastern United States. Students were asked if they encountered struggles and whether they were able to overcome them. Based on how students responded, they were asked to elaborate on (1) how they persevered without struggle, (2) how they were able to overcome their struggles, or (3) what barriers they encountered that did not allow them to overcome their struggles. Each open-ended response was thematically coded to address salient patterns in students’ ability to either persevere or overcome their struggle. We found that during the transition to remote learning, 67% of students experienced struggle. The most reported struggles included: shifts in class format, effective study habits, time management, and increased external commitments. Approximately, 83% of those struggling students were able to overcome their struggle, most often citing their instructor’s support and resources offered during the transition as reasons for their success. Students also cited changes in study habits, and increased confidence or belief that they could excel within the course as ways in which they overcame their struggles. Overall, we found no link between struggles in the classroom and any demographic variables we measured, which included race/ethnicity, gender expression, first-generation college students, transfer student status, and commuter student status. Our results highlight the critical role that instructors play in supporting student learning during these uncertain times by promoting student self-efficacy and positive-growth mindset, providing students with the resources they need to succeed, and creating a supportive and transparent learning environment. 
    more » « less
  5. null (Ed.)
    Online forums are an integral part of modern day courses, but motivating students to participate in educationally beneficial discussions can be challenging. Our proposed solution is to initialize (or “seed”) a new course forum with comments from past instances of the same course that are intended to trigger discussion that is beneficial to learning. In this work, we develop methods for selecting high-quality seeds and evaluate their impact over one course instance of a 186-student biology class. We designed a scale for measuring the “seeding suitability” score of a given thread (an opening comment and its ensuing discussion). We then constructed a supervised machine learning (ML) model for predicting the seeding suitability score of a given thread. This model was evaluated in two ways: first, by comparing its performance to the expert opinion of the course instructors on test/holdout data; and second, by embedding it in a live course, where it was actively used to facilitate seeding by the course instructors. For each reading assignment in the course, we presented a ranked list of seeding recommendations to the course instructors, who could review the list and filter out seeds with inconsistent or malformed content. We then ran a randomized controlled study, in which one group of students was shown seeds that were recommended by the ML model, and another group was shown seeds that were recommended by an alternative model that ranked seeds purely by the length of discussion that was generated in previous course instances. We found that the group of students that received posts from either seeding model generated more discussion than a control group in the course that did not get seeded posts. Furthermore, students who received seeds selected by the ML-based model showed higher levels of engagement, as well as greater learning gains, than those who received seeds ranked by length of discussion. 
    more » « less