An official website of the United States government
Prior research has shown that digital games can enhance STEM education by providing learners with immersive and authentic scientific experiences. However, optimizing the learning outcomes of students engaged in game-based environments requires aligning the game design with diverse student needs. Therefore, an in-depth understanding of player behavior is crucial for identifying students who need additional support or modifications to the game design. This study applies an Ordered Network Analysis (ONA)—a specific kind of Epistemic Network Analysis (ENA)—to examine the game trace log data of student interactions, to gain insights into how learning gains relate to the different ways that students move through an open-ended virtual world for learning microbiology. Our findings reveal that differences between students with high and low learning gains are mediated by their prior knowledge. Specifically, level of prior knowledge is related to behaviors that resemble wheel-spinning, which warrant the development of future interventions. Results also have implications for discovery with modeling approaches and for enhancing in-game support for learners and improving game design.
more »
« less
Recognizing Multi-Party Epistemic Dialogue Acts During Collaborative Game-Based Learning Using Large Language Models
Abstract Understanding students’ multi-party epistemic and topic based-dialogue contributions, or how students present knowledge in group-based chat interactions during collaborative game-based learning, offers valuable insights into group dynamics and learning processes. However, manually annotating these contributions is labor-intensive and challenging. To address this, we develop an automated method for recognizing dialogue acts from text chat data of small groups of middle school students interacting in a collaborative game-based learning environment. Our approach utilizes dual contrastive learning and label-aware data augmentation to fine-tune large language models’ underlying embedding representations within a supervised learning framework for epistemic and topic-based dialogue act classification. Results show that our method achieves a performance improvement of 4% to 8% over baseline methods in two key classification scenarios. These findings highlight the potential for automated dialogue act recognition to support understanding of how meaning-making occurs by focusing on the development and evolution of knowledge in group discourse, ultimately providing teachers with actionable insights to better support student learning.
more »
« less
- Award ID(s):
- 2112635
- PAR ID:
- 10560248
- Publisher / Repository:
- International Journal of Artificial Intelligence in Education
- Date Published:
- Journal Name:
- International Journal of Artificial Intelligence in Education
- ISSN:
- 1560-4292
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
rior research has shown that digital games can enhance STEM education by providing learners with immersive and authentic scientific experiences. However, optimizing the learning outcomes of students engaged in game-based environments requires aligning the game design with diverse student needs. Therefore, an in-depth understanding of player behavior is crucial for identifying students who need additional support or modifications to the game design. This study applies an Ordered Network Analysis (ONA)—a specific kind of Epistemic Network Analysis (ENA)—to examine the game trace log data of student interactions, to gain insights into how learning gains relate to the different ways that students move through an open-ended virtual world for learning microbiology. Our findings reveal that differences between students with high and low learning gains are mediated by their prior knowledge. Specifically, level of prior knowledge is related to behaviors that resemble wheel-spinning, which warrant the development of future interventions. Results also have implications for discovery with modeling approaches and for enhancing in-game support for learners and improving game design.more » « less
-
Successful problem-based learning (PBL) often requires students to collectively regulate their learning processes as a group and engage in socially shared regulation of learning (SSRL). This paper focuses on how facilitators supported SSRL in the context of middle-school game-based PBL. Using conversation analysis, this study analyzed text-based chat messages of facilitators and students collected during gameplay. The analysis revealed direct modeling strategies such as performing regulative processes, promoting group awareness, and dealing with contingency as well as indirect strategies including prompting questions and acknowledgment of regulation, and the patterns of how facilitation faded to yield responsibilities to students to regulate their own learning. The findings will inform researchers and practitioners to design prompts and develop technological tools such as adaptive scaffolding to support SSRL in PBL or other collaborative inquiry processes.more » « less
-
Zhai, X; Latif, E; Liu, N; Biswas, G; Yin, Y (Ed.)Collaborative dialogue offers rich insights into students’ learning and critical thinking, which is essential for personalizing pedagogical agent interactions in STEM+C settings. While large language models (LLMs) facilitate dynamic pedagogical interactions, hallucinations undermine confidence, trust, and instructional value. Retrieval-augmented generation (RAG) grounds LLM outputs in curated knowledge, but requires a clear semantic link between user input and a knowledge base, which is often weak in student dialogue. We propose log-contextualized RAG (LC-RAG), which enhances RAG retrieval by using the environment logs to contextualize collaborative discourse. Our findings show that LCRAG improves retrieval over a discourse-only baseline and allows our collaborative peer agent, Copa, to deliver relevant, personalized guidance that supports students’ critical thinking and epistemic decision-making in a collaborative computational modeling environment, C2STEM.more » « less
-
Mills, Caitlin; Alexandron, Giora; Taibi, Davide; Lo_Bosco, Giosuè; Paquette, Luc (Ed.)Understanding the relationship between student behaviors and learning outcomes is crucial for designing effective collaborative learning environments. However, collaborative learning analytics poses significant challenges, not only due to the complex interplay between collaborative problem-solving and collaborative dialogue but also due to the need for model interpretability. To address these challenges, this paper introduces a multi-view predictive student modeling framework using causal graph discovery. We first extract interpretable behavioral features from students' collaborative dialogue data and game trace logs to predict student learning within a collaborative game-based learning environment. We then apply constraint-based sequential pattern mining to identify cognitive and social behavioral patterns in student's data to improve predictive power. We employ unified causal modeling for interpreting model outputs, using causal discovery methods to reveal key interactions among student behaviors that significantly contribute to predicting learning outcomes and identifying frequent collaborative problem-solving skills. Evaluations of the predictive student modeling framework show that combining features from dialogue and in-game behaviors improves the prediction of student learning gains. The findings highlight the potential of multi-view behavioral data and causal analysis to improve both the effectiveness and the interpretability of collaborative learning analytics.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript
- Journal Article:
- https://doi.org/10.1007/s40593-024-00436-8
