skip to main content


This content will become publicly available on July 1, 2024

Title: Scalable and Equitable Math Problem Solving Strategy Prediction in Big Educational Data
Understanding a student's problem-solving strategy can have a significant impact on effective math learning using Intelligent Tutoring Systems (ITSs) and Adaptive Instructional Systems (AISs). For instance, the ITS/AIS can better personalize itself to correct specific misconceptions that are indicated by incorrect strategies, specific problems can be designed to improve strategies and frustration can be minimized by adapting to a student's natural way of thinking rather than trying to fit a standard strategy for all. While it may be possible for human experts to identify strategies manually in classroom settings with sufficient student interaction, it is not possible to scale this up to big data. Therefore, we leverage advances in Machine Learning and AI methods to perform scalable strategy prediction that is also fair to students at all skill levels. Specifically, we develop an embedding called MVec where we learn a representation based on the mastery of students. We then cluster these embeddings with a non-parametric clustering method where each cluster contains instances that have approximately symmetrical strategies. The strategy prediction model is trained on instances sampled from these clusters ensuring that we train the model over diverse strategies. Using real world large-scale student interaction datasets from MATHia, we show that our approach can scale up to achieve high accuracy by training on a small sample of a large dataset and also has predictive equality, i.e., it can predict strategies equally well for learners at diverse skill levels.  more » « less
Award ID(s):
1934745 2008812
NSF-PAR ID:
10447599
Author(s) / Creator(s):
; ;
Publisher / Repository:
International Educational Data Mining Society
Date Published:
Journal Name:
Proceedings of the 16th International Conference on Educational Data Mining, International Educational Data Mining Society
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. null (Ed.)
    Engineering instructors often use physical manipulatives such as foam beams, rolling cylinders, and large representations of axis systems to demonstrate mechanics concepts and help students visualize systems. Additional benefits are possible when manipulatives are in the hands of individual students or small teams of students who can explore concepts at their own pace and focus on their specific points of confusion. Online learning modalities require new strategies to promote spatial visualization and kinesthetic learning. Potential solutions include creating videos of the activities, using CAD models to demonstrate the principles, programming computer simulations, and providing hands-on manipulatives to students for at-home use. This Work-in-Progress paper discusses our experiences with this last strategy in statics courses two western community colleges and a western four-year university where we supplied students with their own hands-on kits. We have previously reported on the successful implementation of a hands-on statics kit consisting of 3D printed components and standard hardware. The kit was originally designed for use by teams of students during class to engage with topics such as vectors, moments, and rigid body equilibrium. With the onset of the COVID-19 pandemic and shift to online instruction, the first author developed a scaled down version of the kit for at-home use by individual students and modified the associated activity worksheets accordingly. For the community college courses, local students picked up their models at the campus bookstore. We also shipped some of the kits to students who were unable to come to campus, including some in other countries. Due to problems with printing and availability of materials, only 18 kits were available for the class of 34 students at the university implementation. Due to this circumstance, students were placed in teams and asked to work together virtually, one student showing the kit to the other student as they worked through the worksheet prompts. One community college instructor took this approach as well for a limited number of international students who did not receive their kits in a timely manner due to shipping problems. Two instructors assigned the hands-on kits as asynchronous learning activities in their respective online courses, with limited guidance on their use. The third used the kits primarily in synchronous online class meetings. We found that students’ reaction to the models varied by pilot site and presume that implementation differences contributed to this variation. In all cases, student feedback was less positive than it has been for face-to-face courses that used the models from which the take home kit was adapted. Our main conclusion is that implementation matters. Doing hands-on learning in an online course requires some fundamental rethinking about how the learning is structured and scaffolded. 
    more » « less
  2. Abstract  
    more » « less
  3. Academic makerspaces have continued to rise in popularity as research shows the diverse benefits they provide to students. More and more engineering curriculums are incorporating makerspaces and as such there is a need to better understand how their student users can best be served. Surveys administered to makerspace users at a public research university in the Southwest United States during Fall 2020, Spring 2021, Spring 2022, and Fall 2022 tracked student tool usage trends with academic career stages. The survey asked questions about prior experience, motivation, tool usage, and demographics. Analyzed results for mechanical engineering student users provide insight into how curriculum and class year affect the specific tools used and the percentage of students who used a particular tool. The survey results also create a bipartite network model of students and tools, mimicking plant-pollinator type mutualistic networks in ecology. The bipartite network models the student interactions with the tools and visualizes how students interact with the tools. This network modeling enables ecological network analysis techniques to identify key makerspace actors quantitatively. Ecological modularity, for example, identifies divisions in the student-tool makerspace network that highlight how students from different majors (here we investigate mechanical) utilize the makerspace's tools. Modularity is also able to identify “hub” tools in the space, defined as tools central to a student's interaction within the space, based on student-tool connectivity data. The analysis finds that tools commonly used for class by mechanical engineering students, such as the 3D printer or laser cutter, act as gateway tools that bring users into the space and help spark interest in the space's other tools. Using the combined insights from the survey results and the network analysis, ecological network metrics are shown here to be a promising route for informing makerspace policy, tool purchases, and curriculum development. The results can help ensure that the space is set up to give students the best learning opportunities. 
    more » « less
  4. Several studies have highlighted the positive effects that active learning may have on student engagement and performance. However, the influence of active learning strategies is mediated by several factors, including the nature of the learning environment and the cognitive level of in-class tasks. These factors can affect different dimensions of student engagement such as the nature of social processing in student groups, how knowledge is used and elaborated upon by students during in-class tasks, and the amount of student participation in group activities. In this study involving four universities in the US, we explored the association between these different dimensions of student engagement and the cognitive level of assigned tasks in five distinct general chemistry learning environments where students were engaged in group activities in diverse ways. Our analysis revealed a significant association between task level and student engagement. Retrieval tasks often led to a significantly higher number of instances of no interaction between students and individualistic work, and a lower number of knowledge construction and collaborative episodes with full student participation. Analysis tasks, on the other hand, were significantly linked to more instances of knowledge construction and collaboration with full group participation. Tasks at the comprehension level were distinctive in their association with more instances of knowledge application and multiple types of social processing. The results of our study suggest that other factors such as the nature of the curriculum, task timing, and class setting may also affect student engagement during group work. 
    more » « less
  5. No skill is more important for a student of mechanics than the ability to draw a complete and accurate free-body diagram (FBD). A good FBD facilitates proper accounting of forces when writing the balances that lead to governing equations in statics, solid mechanics, and dynamics. Because this skill is essential, educational approaches that improve the ability of students to draw correct FBDs are critical for maximizing the potential of the next generation of engineers. Traditionally, learning to draw FBDs involves classroom instruction followed by homework practice consisting of problems drawn from a textbook. Homework as practice does not serve all students well, because it does not scaffold the process of drawing FBDs in terms of distinct tasks (e.g., isolating the body, considering support reactions) nor does it offer immediate feedback, which students often need to avoid falling into the same error repeatedly. To address these shortcomings, we embarked on the design, implementation, and testing of a mobile application (app) that offers an alternative venue for FBD practice. The app provides students with asynchronous opportunities for training, varied tasks that target specific FBD issues, and several levels of immediate feedback. We hypothesize that the gamified environment and puzzle-based gameplay will improve student skill and self-efficacy in drawing FBDs, particularly for women, who may feel less confident in their spatial skills. Data collected to describe student experiences may also provide additional insight into how to improve FBD instruction generally. In this paper, we detail the process for designing and implementing the app and provide initial data regarding student impressions and use. The app was piloted in Fall 2022 in a large Introduction to Statics course as a non-graded study activity; all students except one (n=97) participated in an evaluation of its design features and user experiences. Approximately half (54%) of students indicated they had played half or more of the available games. When commenting about how the FBD app did, or did not, help their learning, 49% of respondents appreciated that the app allowed additional opportunities for practice. Students used these opportunities to further develop several skills, such as visualizing the system and setting up accurate diagrams, which strengthened their confidence and reviewed key concepts. While describing the value of practicing through the app, 21% of students called out how the app provided feedback. They specifically mentioned the positive experiences of receiving feedback that is immediate, that explains boundary connections, and that deepens learning after mistakes are made. These and other findings from the pilot study are discussed with corresponding next steps for development. 
    more » « less