Prior research has shown that digital games can enhance STEM education by providing learners with immersive and authentic scientific experiences. However, optimizing the learning outcomes of students engaged in game-based environments requires aligning the game design with diverse student needs. Therefore, an in-depth understanding of player behavior is crucial for identifying students who need additional support or modifications to the game design. This study applies an Ordered Network Analysis (ONA)—a specific kind of Epistemic Network Analysis (ENA)—to examine the game trace log data of student interactions, to gain insights into how learning gains relate to the different ways that students move through an open-ended virtual world for learning microbiology. Our findings reveal that differences between students with high and low learning gains are mediated by their prior knowledge. Specifically, level of prior knowledge is related to behaviors that resemble wheel-spinning, which warrant the development of future interventions. Results also have implications for discovery with modeling approaches and for enhancing in-game support for learners and improving game design.
more »
« less
Recognizing Multi-Party Epistemic Dialogue Acts During Collaborative Game-Based Learning Using Large Language Models
Understanding students’ multi-party epistemic and topic based-dialogue contributions, or how students present knowledge in group-based chat interactions during collaborative game-based learning, offers valuable insights into group dynamics and learning processes. However, manually annotating these contributions is labor-intensive and challenging. To address this, we develop an automated method for recognizing dialogue acts from text chat data of small groups of middle school students interacting in a collaborative game-based learning environment. Our approach utilizes dual contrastive learning and label-aware data augmentation to fine-tune large language models’ underlying embedding representations within a supervised learning framework for epistemic and topic-based dialogue act classification. Results show that our method achieves a performance improvement of 4% to 8% over baseline methods in two key classification scenarios. These findings highlight the potential for automated dialogue act recognition to support understanding of how meaning-making occurs by focusing on the development and evolution of knowledge in group discourse, ultimately providing teachers with actionable insights to better support student learning.
more » « less- Award ID(s):
- 2112635
- PAR ID:
- 10555091
- Publisher / Repository:
- Springer Science + Business Media
- Date Published:
- Journal Name:
- International Journal of Artificial Intelligence in Education
- ISSN:
- 1560-4292
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
rior research has shown that digital games can enhance STEM education by providing learners with immersive and authentic scientific experiences. However, optimizing the learning outcomes of students engaged in game-based environments requires aligning the game design with diverse student needs. Therefore, an in-depth understanding of player behavior is crucial for identifying students who need additional support or modifications to the game design. This study applies an Ordered Network Analysis (ONA)—a specific kind of Epistemic Network Analysis (ENA)—to examine the game trace log data of student interactions, to gain insights into how learning gains relate to the different ways that students move through an open-ended virtual world for learning microbiology. Our findings reveal that differences between students with high and low learning gains are mediated by their prior knowledge. Specifically, level of prior knowledge is related to behaviors that resemble wheel-spinning, which warrant the development of future interventions. Results also have implications for discovery with modeling approaches and for enhancing in-game support for learners and improving game design.more » « less
-
more » « less
Abstract This exploratory paper highlights how problem‐based learning (PBL) provided the pedagogical framework used to design and interpret learning analytics from C
rystal Island: EcoJourneys , a collaborative game‐based learning environment centred on supporting science inquiry. In Crystal Island: EcoJourneys , students work in teams of four, investigate the problem individually and then utilize a brainstorming board, an in‐game PBL whiteboard that structured the collaborative inquiry process. The paper addresses a central question: how can PBL support the interpretation of the observed patterns in individual actions and collaborative interactions in the collaborative game‐based learning environment? Drawing on a mixed method approach, we first analyzed students' pre‐ and post‐test results to determine if there were learning gains. We then used principal component analysis (PCA) to describe the patterns in game interaction data and clustered students based on the PCA. Based on the pre‐ and post‐test results and PCA clusters, we used interaction analysis to understand how collaborative interactions unfolded across selected groups. Results showed that students learned the targeted content after engaging with the game‐based learning environment. Clusters based on the PCA revealed four main ways of engaging in the game‐based learning environment: students engaged in low to moderate self‐directed actions with (1) high and (2) moderate collaborative sense‐making actions, (3) low self‐directed with low collaborative sense‐making actions and (4) high self‐directed actions with low collaborative sense‐making actions. Qualitative interaction analysis revealed that a key difference among four groups in each cluster was the nature of verbal student discourse: students in the low to moderate self‐directed and high collaborative sense‐making cluster actively initiated discussions and integrated information they learned to the problem, whereas students in the other clusters required more support. These findings have implications for designing adaptive support that responds to students' interactions with in‐game activities.Practitioner notes What is already known about this topic
Learning analytic methods have been effective for understanding student learning interactions for the purposes of assessment, profiling student behaviour and the effectiveness of interventions.
However, the interpretation of analytics from these diverse data sets are not always grounded in theory and challenges of interpreting student data are further compounded in collaborative inquiry settings, where students work in groups to solve a problem.
What this paper adds
Problem‐based learning as a pedagogical framework allowed for the design to focus on individual and collaborative actions in a game‐based learning environment and, in turn, informed the interpretation of game‐based analytics as it relates to student's self‐directed learning in their individual investigations and collaborative inquiry discussions.
The combination of principal component analysis and qualitative interaction analysis was critical in understanding the nuances of student collaborative inquiry.
Implications for practice and/or policy
Self‐directed actions in individual investigations are critical steps to collaborative inquiry. However, students may need to be encouraged to engage in these actions.
Clustering student data can inform which scaffolds can be delivered to support both self‐directed learning and collaborative inquiry interactions.
All students can engage in knowledge‐integration discourse, but some students may need more direct support from teachers to achieve this.
-
more » « less
Background. Middle school students’ math anxiety and low engagement have been major issues in math education. In order to reduce their anxiety and support their math learning, game-based learning (GBL) has been implemented. GBL research has underscored the role of social dynamics to facilitate a qualitative understanding of students’ knowledge. Whereas students’ peer interactions have been deemed a social dynamic, the relationships among peer interaction, task efficiency, and learning engagement were not well understood in previous empirical studies.
Method. This mixed-method research implemented E-Rebuild, which is a 3D architecture game designed to promote students’ math problem-solving skills. We collected a total of 102 50-minutes gameplay sessions performed by 32 middle school students. Using video-captured and screen-recorded gameplaying sessions, we implemented behavior observations to measure students’ peer interaction efficiency, task efficiency, and learning engagement. We used association analyses, sequential analysis, and thematic analysis to explain how peer interaction promoted students’ task efficiency and learning engagement.
Results. Students’ peer interactions were negatively related to task efficiency and learning engagement. There were also different gameplay patterns by students’ learning/task-relevant peer-interaction efficiency (PIE) level. Students in the low PIE group tended to progress through game tasks more efficiently than those in the high PIE group. The results of qualitative thematic analysis suggested that the students in the low PIE group showed more reflections on game-based mathematical problem solving, whereas those with high PIE experienced distractions during gameplay.
Discussion. This study confirmed that students’ peer interactions without purposeful and knowledge-constructive collaborations led to their low task efficiency, as well as low learning engagement. The study finding shows further design implications: (1) providing in-game prompts to stimulate students’ math-related discussions and (2) developing collaboration contexts that legitimize students’ interpersonal knowledge exchanges with peers.
-
more » « less
Abstract Engineering design provides unique ways to include epistemic tools to support collaborative sense‐making, reasoning with evidence, and assessing knowledge. Engineering design processes often require students to apply science concepts to solve problems. We draw from five engineering curricular units that engaged students in specific epistemic practices of engineering: constructing models and prototypes, making trade‐offs between criteria and constraints, and communicating through uses of conventionalized verbal, written, and symbolic models. Through analysis of curriculum products, student artifacts, and classroom discourse, we show how engaging in such practices requires the use of epistemic tools that shape, and are shaped by, the knowledge construction work of the members of the classrooms. The epistemic tools foster creating, sharing, and assessing knowledge claims. Six principles of practice for education demonstrate how such tools can be educative. These principles evince how epistemic tools support goal‐directed, concerted activity that can support the learning of disciplinary knowledge and practice and offer the potential to increase student agency.
