skip to main content


Title: Demographic Mediation of the Relationship Between Engagement and Performance in a Blended Dynamics Engineering Course
ABSTRACT CONTEXT This paper examines an engineering dynamics course at Purdue University that was specifically designed to create an active, blended, and collaborative environment. In addition to in-person classes and support, students have access to blended content such as solution videos, mechanics visualizations, a course discussion forum, and interactive simulations. PURPOSE Many studies have shown that students’ engagement in an online discussion forum enhances their learning performance (Davies & Graff, 2005; Hrastinski, 2008). However, our previous research showed that students’ engagement in the online forum of our dynamics course differed significantly across students’ demographics. We showed that women, white, or Asian American students were more likely to be involved in online discussions than men, international, or Hispanic students (Duan et al., 2018). In this paper, we take the previous analysis further by examining whether the observed differences in online student engagement mediate or moderate student performance. APPROACH To answer our research question, we will first investigate the mediation effect by creating two models. A first model with race/international status as the mediating variable and gender identity as a control variable, and a second model with gender identity as the mediating variable and race/international status as a control. Second, we will investigate the moderation effect of demographic factors by creating a regression model including interaction terms to show the relationship of each demographic’s discussion forum engagement to overall performance. The goal of investigating these interaction terms is to determine if a moderating relationship exists where demographic factors impact online engagement, which in turn impact course performance. CONCLUSIONS We find that gender identity is the only significant demographic factor that moderates the effect of a student’s engagement on their performance. Based on the findings of our previous work, students of various racial and ethnic identities do engage differently in the discussion forum. However, this analysis was unable to detect any significant difference in student engagement based on demographics. Our paper contributes to understanding the mechanisms through which students’ engagement can translate into academic performance by focusing on their demographic background. The moderating role of students’ demographic background calls for a more targeted design of instructional tools in blended and collaborative environments to better support students from various demographic backgrounds.  more » « less
Award ID(s):
1915574
PAR ID:
10355868
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
Proceedings of REES AAEE 2021
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. null (Ed.)
    Students belonging to marginalized groups experience positive impacts when taught by a teacher of the same race, ethnicity, and gender. The unique nature of standardized, asynchronous online course taking allows for greater separation of any possible educational benefits of student versus teacher-driven mechanisms contributing to these improved outcomes. Using a student-by-course fixed effect strategy on data from a large urban school district, I examined associations between whether students experienced racial/ethnic or gender congruence with their remote instructor and both engagement and learning outcomes. Students who identified as Black demonstrated higher rates of engagement, although no difference in achievement, within lessons taught by a same-race remote instructor. I find that representation is associated with engagement even when instructors follow closely scripted lessons, representation occurs in only small doses, and instruction occurs in an impersonal setting. 
    more » « less
  2. This study investigated patterns in the development of computational thinking practices in the context of the Exploring Computer Science (ECS) program, a high school introductory CS course and professional development program designed to foster deep engagement through equitable inquiry around CS concepts. Past research indicates that the personal relevance of the ECS experience influences students' expectancy-value towards computer science. Expectancy-value is a construct that is predictive of career choices. We extended our research to examine whether expectancy-value influences the development of computational thinking practices. This study took place in the context of two ECS implementation projects across two states. Twenty teachers, who implemented ECS in 2016–17, participated in the research. There were 906 students who completed beginning and end of year surveys and assessments. The surveys included demographic questions, a validated expectancy-value scale, and questions about students' course experiences. The assessments were developed and validated by SRI International as a companion to the ECS course. Overall, student performance statistically increased from pretest to posttest with effect size of 0.74. There were no statistically significant differences in performance by gender or race/ethnicity. These results are consistent with earlier findings that a personally relevant course experience positively influences students' expectancy for success. These results expanded on prior research by indicating that students' expectancy-value for computer science positively influenced student learning. 
    more » « less
  3. null (Ed.)
    Online forums are an integral part of modern day courses, but motivating students to participate in educationally beneficial discussions can be challenging. Our proposed solution is to initialize (or “seed”) a new course forum with comments from past instances of the same course that are intended to trigger discussion that is beneficial to learning. In this work, we develop methods for selecting high-quality seeds and evaluate their impact over one course instance of a 186-student biology class. We designed a scale for measuring the “seeding suitability” score of a given thread (an opening comment and its ensuing discussion). We then constructed a supervised machine learning (ML) model for predicting the seeding suitability score of a given thread. This model was evaluated in two ways: first, by comparing its performance to the expert opinion of the course instructors on test/holdout data; and second, by embedding it in a live course, where it was actively used to facilitate seeding by the course instructors. For each reading assignment in the course, we presented a ranked list of seeding recommendations to the course instructors, who could review the list and filter out seeds with inconsistent or malformed content. We then ran a randomized controlled study, in which one group of students was shown seeds that were recommended by the ML model, and another group was shown seeds that were recommended by an alternative model that ranked seeds purely by the length of discussion that was generated in previous course instances. We found that the group of students that received posts from either seeding model generated more discussion than a control group in the course that did not get seeded posts. Furthermore, students who received seeds selected by the ML-based model showed higher levels of engagement, as well as greater learning gains, than those who received seeds ranked by length of discussion. 
    more » « less
  4. Coffman, Clark (Ed.)
    Instructors use a variety of online formative assessment (FA) activities to support learning outside class. Previous studies have revealed barriers for students in online courses, but little is known about the barriers students experience when completing online FA assignments. Understanding these barriers to access is critical to fostering more inclusive learning for all students. Using a framework from previous work in online learning, we examined student perceptions of online FA access with respect to five barrier categories: technical resources, instructor organization, social interactions, personal engagement, and learning environment. We developed and administered a survey to more than 1200 undergraduate biology students at 2-year and 4-year institutions. Students responded to statements using Likert scales and open-ended prompts. Statistical models indicated differences in access across the barrier categories and revealed that demographic characteristics were associated with certain barrier categories. Furthermore, technical resources, instructor organization, and personal engagement barriers were associated with lower course performance. In open-ended responses, students most frequently suggested that changes to scheduling logistics, course delivery, and FA format would improve their online FA experience. We discuss how these findings and student suggestions can inform instruction, particularly how instructors can alter their FA characteristics to better suit their student populations. 
    more » « less
  5. null (Ed.)
    Background - One of the most critical challenges in engineering education is improving students’ divergent thinking skills. Usually, we observe students’ fixating on only one single solution for engineering problems. However, their ability to think outside the box and provide alternative solutions should be developed. Research shows that engagement may foster the development of thoughts and boost creativity. Purpose/Hypothesis – Our aim was to investigate students’ engagement with tasks that inspire different facets of creativity (verbal, numeric, and visual). Considering the role of demographics in student engagement, we explored the relationship between their engagement level and demographic traits such as gender, major, age, grades (GPA), and the languages they know besides their native tongue. Design/Method - We utilized electrodermal activity (EDA) sensors, a well-documented proxy of emotional engagement, to measure students’ engagement level while performing tasks that inspire different facets of creativity (verbal, numeric, and visual). Due to the non-normal distribution of the data, non-parametric statistical tests were conducted considering engagement as a dependent variable and demographic traits as independent variables. Results - Statistically significant differences in students’ engagement when exposed to creativity inspired tasks were observed. However, no association between demographics and engagement levels were detected. Conclusions - The results of the study may support educators in designing the instructional materials considering creativity-inspired activities so that students’ engagement level can be increased. Further, results from this study can inform experimental designs, specifically participant selection, in engagement focused studies. 
    more » « less