An official website of the United States government
Understanding the thought processes of students as they progress from initial (incorrect) answers toward correct answers is a challenge for instructors, both in this pandemic and beyond. This paper presents a general network visualization learning analytics system that helps instructors to view a sequence of answers input by students in a way that makes student learning progressions apparent. The system allows instructors to study individual and group learning at various levels of granularity. The paper illustrates how the visualization system is employed to analyze student responses collected through an intervention. The intervention is BeginToReason, an online tool that helps students learn and use symbolic reasoning-reasoning about code behavior through abstract values instead of concrete inputs. The specific focus is analysis of tool-collected student responses as they perform reasoning activities on code involving conditional statements. Student learning is analyzed using the visualization system and a post-test. Visual analytics highlights include instances where students producing one set of incorrect answers initially perform better than a different set and instances where student thought processes do not cluster well. Post-test data analysis provides a measure of student ability to apply what they have learned and their holistic understanding.
more »
« less
Scalable and Equitable Math Problem Solving Strategy Prediction in Big Educational Data
Understanding a student's problem-solving strategy can have a significant impact on effective math learning using Intelligent Tutoring Systems (ITSs) and Adaptive Instructional Systems (AISs). For instance, the ITS/AIS can better personalize itself to correct specific misconceptions that are indicated by incorrect strategies, specific problems can be designed to improve strategies and frustration can be minimized by adapting to a student's natural way of thinking rather than trying to fit a standard strategy for all. While it may be possible for human experts to identify strategies manually in classroom settings with sufficient student interaction, it is not possible to scale this up to big data. Therefore, we leverage advances in Machine Learning and AI methods to perform scalable strategy prediction that is also fair to students at all skill levels. Specifically, we develop an embedding called MVec where we learn a representation based on the mastery of students. We then cluster these embeddings with a non-parametric clustering method where each cluster contains instances that have approximately symmetrical strategies. The strategy prediction model is trained on instances sampled from these clusters ensuring that we train the model over diverse strategies. Using real world large-scale student interaction datasets from MATHia, we show that our approach can scale up to achieve high accuracy by training on a small sample of a large dataset and also has predictive equality, i.e., it can predict strategies equally well for learners at diverse skill levels.
more »
« less
- PAR ID:
- 10447599
- Publisher / Repository:
- International Educational Data Mining Society
- Date Published:
- Journal Name:
- Proceedings of the 16th International Conference on Educational Data Mining, International Educational Data Mining Society
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Roll, I; McNamara, D; Sosnovsky, S; Luckin, R; Dimitrova, V. (Ed.)Knowledge tracing refers to a family of methods that estimate each student’s knowledge component/skill mastery level from their past responses to questions. One key limitation of most existing knowledge tracing methods is that they can only estimate an overall knowledge level of a student per knowledge component/skill since they analyze only the (usually binary-valued) correctness of student responses. Therefore, it is hard to use them to diagnose specific student errors. In this paper, we extend existing knowledge tracing methods beyond correctness prediction to the task of predicting the exact option students select in multiple choice questions. We quantitatively evaluate the performance of our option tracing methods on two large-scale student response datasets. We also qualitatively evaluate their ability in identifying common student errors in the form of clusters of incorrect options across different questions that correspond to the same error.more » « less
-
Several studies have highlighted the positive effects that active learning may have on student engagement and performance. However, the influence of active learning strategies is mediated by several factors, including the nature of the learning environment and the cognitive level of in-class tasks. These factors can affect different dimensions of student engagement such as the nature of social processing in student groups, how knowledge is used and elaborated upon by students during in-class tasks, and the amount of student participation in group activities. In this study involving four universities in the US, we explored the association between these different dimensions of student engagement and the cognitive level of assigned tasks in five distinct general chemistry learning environments where students were engaged in group activities in diverse ways. Our analysis revealed a significant association between task level and student engagement. Retrieval tasks often led to a significantly higher number of instances of no interaction between students and individualistic work, and a lower number of knowledge construction and collaborative episodes with full student participation. Analysis tasks, on the other hand, were significantly linked to more instances of knowledge construction and collaboration with full group participation. Tasks at the comprehension level were distinctive in their association with more instances of knowledge application and multiple types of social processing. The results of our study suggest that other factors such as the nature of the curriculum, task timing, and class setting may also affect student engagement during group work.more » « less
-
Undergraduate Computing Tutors' Perceptions of their Roles, Stressors, and Barriers to EffectivenessUndergraduate teaching assistants (tutors) are commonly employed in computing courses to help students with programming assignments. Prior research in computing education has reported the benefits of tutoring both for students and for the tutors' own learning. In contrast, recent research that examined actual tutoring sessions has reported that these sessions may be less productive than one might hope, with tutors often just giving students the answers to their problems without trying to teach the underlying concepts. To better understand why tutors may be employing these suboptimal practices, we interviewed ten tutors across early computing courses in higher education to identify their perceived role in these sessions, what stressors and factors influence their ability to perform their job effectively, and what kinds of best practices they learned in their tutor training course. Tutors reported their roles around student learning, gauging student understanding, identifying or providing solutions to students, and providing socioemotional support. They reported their stressors around environmental factors (e.g., number of students waiting to be helped, preparation time, peer-tutor frustrations), internal influences, student behavior, student skill levels, and feeling the need to ''read a student's mind.'' Regarding their tutor training course, Tutors reported learning about interaction guidelines and procedures and question-based problem solving. We conclude by discussing how these results may contribute to the less-effective behaviors seen in prior research and potential ways to improve tutoring in computing courses.more » « less
-
Academic makerspaces have continued to rise in popularity as research shows the diverse benefits they provide to students. More and more engineering curriculums are incorporating makerspaces and as such there is a need to better understand how their student users can best be served. Surveys administered to makerspace users at a public research university in the Southwest United States during Fall 2020, Spring 2021, Spring 2022, and Fall 2022 tracked student tool usage trends with academic career stages. The survey asked questions about prior experience, motivation, tool usage, and demographics. Analyzed results for mechanical engineering student users provide insight into how curriculum and class year affect the specific tools used and the percentage of students who used a particular tool. The survey results also create a bipartite network model of students and tools, mimicking plant-pollinator type mutualistic networks in ecology. The bipartite network models the student interactions with the tools and visualizes how students interact with the tools. This network modeling enables ecological network analysis techniques to identify key makerspace actors quantitatively. Ecological modularity, for example, identifies divisions in the student-tool makerspace network that highlight how students from different majors (here we investigate mechanical) utilize the makerspace's tools. Modularity is also able to identify “hub” tools in the space, defined as tools central to a student's interaction within the space, based on student-tool connectivity data. The analysis finds that tools commonly used for class by mechanical engineering students, such as the 3D printer or laser cutter, act as gateway tools that bring users into the space and help spark interest in the space's other tools. Using the combined insights from the survey results and the network analysis, ecological network metrics are shown here to be a promising route for informing makerspace policy, tool purchases, and curriculum development. The results can help ensure that the space is set up to give students the best learning opportunities.more » « less
