skip to main content


Title: Data-Driven Modeling of Learners’ Individual Differences for Predicting Engagement and Success in Online Learning
Individual differences have been recognized as an important factor in the learning process. However, there are few successes in using known dimensions of individual differences in solving an important problem of predicting student performance and engagement in online learning. At the same time, learning analytics research has demonstrated that the large volume of learning data collected by modern e-learning systems could be used to recognize student behavior patterns and could be used to connect these patterns with measures of student performance. Our paper attempts to bridge these two research directions. By applying a sequence mining approach to a large volume of learner data collected by an online learning system, we build models of student learning behavior. However, instead of following modern work on behavior mining (i.e., using this behavior directly for performance prediction tasks), we attempt to follow traditional work on modeling individual differences in quantifying this behavior on a latent data-driven personality scale. Our research shows that this data-driven model of individual differences performs significantly better than several traditional models of individual differences in predicting important parameters of the learning process, such as success and engagement.  more » « less
Award ID(s):
1740775
NSF-PAR ID:
10300393
Author(s) / Creator(s):
;
Date Published:
Journal Name:
the 29th ACM Conference on User Modeling, Adaptation and Personalization
Page Range / eLocation ID:
201 to 212
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Educational data mining research has demonstrated that the large volume of learning data collected by modern e-learning systems could be used to recognize student behavior patterns and group students into cohorts with similar behavior. However, few attempts have been done to connect and compare behavioral patterns with known dimensions of individual differences. To what extent learner behavior is defined by known individual differences? Which of them could be a better predictor of learner engagement and performance? Could we use behavior patterns to build a data-driven model of individual differences that could be more useful for predicting critical outcomes of the learning process than traditional models? Our paper attempts to answer these questions using a large volume of learner data collected in an online practice system. We apply a sequential pattern mining approach to build individual models of learner practice behavior and reveal latent student subgroups that exhibit considerably different practice behavior. Using these models we explored the connections between learner behavior and both, the incoming and outgoing parameters of the learning process. Among incoming parameters we examined traditionally collected individual differences such as self-esteem, gender, and knowledge monitoring skills. We also attempted to bridge the gap between cluster-based behavior pattern models and traditional scale-based models of individual differences by quantifying learner behavior on a latent data-driven scale. Our research shows that this data-driven model of individual differences performs significantly better than traditional models of individual differences in predicting important parameters of the learning process, such as performance and engagement. 
    more » « less
  2. In March 2020, the global COVID-19 pandemic forced universities across the United States to immediately stop face-to-face activities and transition to virtual instruction. While this transition was not easy for anyone, the shift to online learning was especially difficult for STEM courses, particularly engineering, which has a strong practical/laboratory component. Additionally, underrepresented students (URMs) in engineering experienced a range of difficulties during this transition. The purpose of this paper is to highlight underrepresented engineering students’ experiences as a result of COVID-19. In particular, we aim to highlight stories shared by participants who indicated a desire to share their experience with their instructor. In order to better understand these experiences, research participants were asked to share a story, using the novel data collection platform SenseMaker, based on the following prompt: Imagine you are chatting with a friend or family member about the evolving COVID-19 crisis. Tell them about something you have experienced recently as an engineering student. Conducting a SenseMaker study involves four iterative steps: 1) Initiation is the process of designing signifiers, testing, and deploying the instrument; 2) Story Collection is the process of collecting data through narratives; 3) Sense-making is the process of exploring and analyzing patterns of the collection of narratives; and 4) Response is the process of amplifying positive stories and dampening negative stories to nudge the system to an adjacent possible (Van der Merwe et al. 2019). Unlike traditional surveys or other qualitative data collection methods, SenseMaker encourages participants to think more critically about the stories they share by inviting them to make sense of their story using a series of triads and dyads. After completing their narrative, participants were asked a series of triadic, dyadic, and sentiment-based multiple-choice questions (MCQ) relevant to their story. For one MCQ, in particular, participants were required to answer was “If you could do so without fear of judgment or retaliation, who would you share this story with?” and were given the following options: 1) Family 2) Instructor 3) Peers 4) Prefer not to answer 5) Other. A third of the participants indicated that they would share their story with their instructor. Therefore, we further explored this particular question. Additionally, this paper aims to highlight this subset of students whose primary motivation for their actions were based on Necessity. High-level qualitative findings from the data show that students valued Grit and Perseverance, recent experiences influenced their Sense of Purpose, and their decisions were majorly made based on Intuition. Chi-squared tests showed that there were not any significant differences between race and the desire to share with their instructor, however, there were significant differences when factoring in gender suggesting that gender has a large impact on the complexity of navigating school during this time. Lastly, ~50% of participants reported feeling negative or extremely negative about their experiences, ~30% reported feeling neutral, and ~20% reported feeling positive or extremely positive about their experiences. In the study, a total of 500 micro-narratives from underrepresented engineering students were collected from June – July 2020. Undergraduate and graduate students were recruited for participation through the researchers’ personal networks, social media, and through organizations like NSBE. Participants had the option to indicate who is able to read their stories 1) Everyone 2) Researchers Only, or 3) No one. This work presents qualitative stories of those who granted permission for everyone to read. 
    more » « less
  3. null (Ed.)
    Over the past two decades, educators have used computer-supported collaborative learning (CSCL) to integrate technology with pedagogy to improve student engagement and learning outcomes. Researchers have also explored the diverse affordances of CSCL, its contributions to engineering instruction, and its effectiveness in K-12 STEM education. However, the question of how students use CSCL resources in undergraduate engineering classrooms remains largely unexplored. This study examines the affordances of a CSCL environment utilized in a sophomore dynamics course with particular attention given to the undergraduate engineering students’ use of various CSCL resources. The resources include a course lecturebook, instructor office hours, a teaching assistant help room, online discussion board, peer collaboration, and demonstration videos. This qualitative study uses semi-structured interview data collected from nine mechanical engineering students (four women and five men) who were enrolled in a dynamics course at a large public research university in Eastern Canada. The interviews focused on the individual student’s perceptions of the school, faculty, students, engineering courses, and implemented CSCL learning environment. The thematic analysis was conducted to analyze the transcribed interviews using a qualitative data analysis software (Nvivo). The analysis followed a six step process: (1) reading interview transcripts multiple times and preliminary in vivo codes; (2) conducting open coding by coding interesting or salient features of the data; (3) collecting codes and searching for themes; (4) reviewing themes and creating a thematic map; (5) finalizing themes and their definitions; and (6) compiling findings. This study found that the students’ use of CSCL resources varied depending on the students’ personal preferences, as well as their perceptions of the given resource’s value and its potential to enhance their learning. For example, the dynamics lecturebook, which had been redesigned to encourage problem solving and note-taking, fostered student collaborative problem solving with their peers. In contrast, the professor’s example video solutions had much more of an influence on students’ independent problem-solving processes. The least frequently used resource was the course’s online discussion forum, which could be used as a means of communication. The findings reveal how computer-supported collaborative learning (CSCL) environments enable engineering students to engage in multiple learning opportunities with diverse and flexible resources to both address and to clarify their personal learning needs. This study strongly recommends engineering instructors adapt a CSCL environment for implementation in their own unique classroom context. 
    more » « less
  4. A variety of research studies reveal the advantages of actively engaging students in the learning process through collaborative work in the classroom. However, the complex nature of the learning environment in large college general chemistry courses makes it challenging to identify the different factors that affect students’ cognitive and social engagement while working on in-class tasks. To provide insights into this area, we took a closer look at students’ conversations during in-class activities to characterize typical discourse patterns and expressed chemical thinking in representative student groups in samples collected in five different learning environments across four universities. For this purpose, we adapted and applied a ‘Community of Learners’ (CoL) theoretical perspective to characterize group activity through the analysis of student discourse. Within a CoL perspective, the extent to which a group functions as a community of learners is analyzed along five dimensions including Community of Discourse (CoD), Legitimization of Differences (LoD), Building on Ideas (BoI), Reflective Learning (RL), and Community of Practice (CoP). Our findings make explicit the complexity of analyzing student engagement in large active learning environments where a multitude of variables can affect group work. These include, among others, group size and composition, the cognitive level of the tasks, the types of cognitive processes used to complete tasks, and the motivation and willingness of students to substantively engage in disciplinary reasoning. Our results point to important considerations in the design and implementation of active learning environments that engage more students with chemical ideas at higher levels of reasoning. 
    more » « less
  5. A variety of research studies reveal the advantages of actively engaging students in the learning process through collaborative work in the classroom. However, the complex nature of the learning environment in large college general chemistry courses makes it challenging to identify the different factors that affect students’ cognitive and social engagement while working on in-class tasks. To provide insights into this area, we took a closer look at students’ conversations during in-class activities to characterize typical discourse patterns and expressed chemical thinking in representative student groups in samples collected in five different learning environments across four universities. For this purpose, we adapted and applied a ‘Community of Learners’ (CoL) theoretical perspective to characterize group activity through the analysis of student discourse. Within a CoL perspective, the extent to which a group functions as a community of learners is analyzed along five dimensions including Community of Discourse (CoD), Legitimization of Differences (LoD), Building on Ideas (BoI), Reflective Learning (RL), and Community of Practice (CoP). Our findings make explicit the complexity of analyzing student engagement in large active learning environments where a multitude of variables can affect group work. These include, among others, group size and composition, the cognitive level of the tasks, the types of cognitive processes used to complete tasks, and the motivation and willingness of students to substantively engage in disciplinary reasoning. Our results point to important considerations in the design and implementation of active learning environments that engage more students with chemical ideas at higher levels of reasoning. 
    more » « less