An official website of the United States government
Understanding the thought processes of students as they progress from initial (incorrect) answers toward correct answers is a challenge for instructors, both in this pandemic and beyond. This paper presents a general network visualization learning analytics system that helps instructors to view a sequence of answers input by students in a way that makes student learning progressions apparent. The system allows instructors to study individual and group learning at various levels of granularity. The paper illustrates how the visualization system is employed to analyze student responses collected through an intervention. The intervention is BeginToReason, an online tool that helps students learn and use symbolic reasoning-reasoning about code behavior through abstract values instead of concrete inputs. The specific focus is analysis of tool-collected student responses as they perform reasoning activities on code involving conditional statements. Student learning is analyzed using the visualization system and a post-test. Visual analytics highlights include instances where students producing one set of incorrect answers initially perform better than a different set and instances where student thought processes do not cluster well. Post-test data analysis provides a measure of student ability to apply what they have learned and their holistic understanding.
more »
« less
Scalable and Equitable Math Problem Solving Strategy Prediction in Big Educational Data
Understanding a student's problem-solving strategy can have a significant impact on effective math learning using Intelligent Tutoring Systems (ITSs) and Adaptive Instructional Systems (AISs). For instance, the ITS/AIS can better personalize itself to correct specific misconceptions that are indicated by incorrect strategies, specific problems can be designed to improve strategies and frustration can be minimized by adapting to a student's natural way of thinking rather than trying to fit a standard strategy for all. While it may be possible for human experts to identify strategies manually in classroom settings with sufficient student interaction, it is not possible to scale this up to big data. Therefore, we leverage advances in Machine Learning and AI methods to perform scalable strategy prediction that is also fair to students at all skill levels. Specifically, we develop an embedding called MVec where we learn a representation based on the mastery of students. We then cluster these embeddings with a non-parametric clustering method where each cluster contains instances that have approximately symmetrical strategies. The strategy prediction model is trained on instances sampled from these clusters ensuring that we train the model over diverse strategies. Using real world large-scale student interaction datasets from MATHia, we show that our approach can scale up to achieve high accuracy by training on a small sample of a large dataset and also has predictive equality, i.e., it can predict strategies equally well for learners at diverse skill levels.
more »
« less
- PAR ID:
- 10447599
- Publisher / Repository:
- International Educational Data Mining Society
- Date Published:
- Journal Name:
- Proceedings of the 16th International Conference on Educational Data Mining, International Educational Data Mining Society
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Several studies have highlighted the positive effects that active learning may have on student engagement and performance. However, the influence of active learning strategies is mediated by several factors, including the nature of the learning environment and the cognitive level of in-class tasks. These factors can affect different dimensions of student engagement such as the nature of social processing in student groups, how knowledge is used and elaborated upon by students during in-class tasks, and the amount of student participation in group activities. In this study involving four universities in the US, we explored the association between these different dimensions of student engagement and the cognitive level of assigned tasks in five distinct general chemistry learning environments where students were engaged in group activities in diverse ways. Our analysis revealed a significant association between task level and student engagement. Retrieval tasks often led to a significantly higher number of instances of no interaction between students and individualistic work, and a lower number of knowledge construction and collaborative episodes with full student participation. Analysis tasks, on the other hand, were significantly linked to more instances of knowledge construction and collaboration with full group participation. Tasks at the comprehension level were distinctive in their association with more instances of knowledge application and multiple types of social processing. The results of our study suggest that other factors such as the nature of the curriculum, task timing, and class setting may also affect student engagement during group work.more » « less
-
Gardner, Stephanie (Ed.)Anxiety can impact overall performance and persistence in college. Student response systems (SRSs), real-time active-learning technologies used to engage students and gauge their understanding, have been shown to elicit anxiety for some students. Kahoot! is an SRS technology that differs from others in that it involves gamification, the use of gamelike elements. Recent studies have explored the impact of active-learning strategies on student anxiety across different institutions, but there is little known about how Kahoot! impacts student perceived anxiety, especially in comparison with other active-learning strategies. In two complementary yet parallel studies of introductory biology courses at a western research-intensive institution ( n = 694) and a southeastern research-intensive institution ( n = 60), we measured students’ perceived anxiety. We then explored how students were influenced by nongraded Kahoot! play and other elements of instruction. Using previously developed and course-specific pre- and post-course surveys, we found students at both universities agreed that nongraded Kahoot! play caused less anxiety compared with other pedagogical practices, such as working in small groups or reading the textbook. After playing Kahoot!, lower-performing students demonstrated greater engagement and lower levels of anxiety compared with their peers, suggesting that Kahoot! may be a particularly engaging active-learning strategy for these students.more » « less
-
In this work, we propose a video-based transfer learning approach for predicting problem outcomes of students working with an intelligent tutoring system (ITS). By analyzing a student's face and gestures, our method predicts the outcome of a student answering a problem in an ITS from a video feed. Our work is motivated by the reasoning that the ability to predict such outcomes enables tutoring systems to adjust interventions, such as hints and encouragement, and to ultimately yield improved student learning. We collected a large labeled dataset of student interactions with an intelligent online math tutor consisting of 68 sessions, where 54 individual students solved 2,749 problems. We will release this dataset publicly upon publication of this paper. It will be available at https://www.cs.bu.edu/faculty/betke/research/learning/. Working with this dataset, our transfer-learning challenge was to design a representation in the source domain of pictures obtained “in the wild” for the task of facial expression analysis, and transferring this learned representation to the task of human behavior prediction in the domain of webcam videos of students in a classroom environment. We developed a novel facial affect representation and a user-personalized training scheme that unlocks the potential of this representation. We designed several variants of a recurrent neural network that models the temporal structure of video sequences of students solving math problems. Our final model, named ATL-BP for Affect Transfer Learning for Behavior Prediction, achieves a relative increase in mean F -score of 50 % over the state-of-the-art method on this new dataset.more » « less
-
Roll, I; McNamara, D; Sosnovsky, S; Luckin, R; Dimitrova, V. (Ed.)Knowledge tracing refers to a family of methods that estimate each student’s knowledge component/skill mastery level from their past responses to questions. One key limitation of most existing knowledge tracing methods is that they can only estimate an overall knowledge level of a student per knowledge component/skill since they analyze only the (usually binary-valued) correctness of student responses. Therefore, it is hard to use them to diagnose specific student errors. In this paper, we extend existing knowledge tracing methods beyond correctness prediction to the task of predicting the exact option students select in multiple choice questions. We quantitatively evaluate the performance of our option tracing methods on two large-scale student response datasets. We also qualitatively evaluate their ability in identifying common student errors in the form of clusters of incorrect options across different questions that correspond to the same error.more » « less
