Intelligent tutoring systems (ITSs) incorporate pedagogical agents (PAs) to scaffold learners' self‐regulated learning (SRL) via prompts and feedback to promote learners' monitoring and regulation of their cognitive, affective, metacognitive and motivational processes to achieve their (sub)goals. This study examines PAs' effectiveness in scaffolding and teaching SRL during learning with MetaTutor, an ITS on the human circulatory system. Undergraduates ( Most learners struggle to efficiently and effectively use self‐regulated learning (SRL) strategies to attain goals and subgoals. There is a need for SRL to be scaffolded for learners to manage multiple goals and subgoals while learning about complex STEM topics. Intelligent tutoring systems (ITSs) typically incorporate pedagogical agents (PAs) to prompt learners to engage in SRL strategy and provide feedback. There are mixed findings on the effectiveness of PAs in scaffolding learners' SRL. We consider PAs not only scaffolders but also teachers of SRL. Results showed that while PAs encouraged the use of SRL strategies when the content was relevant to subgoals, they did not discourage the use of SRL strategies when the content was not relevant. Results for this study were mixed in their support of PAs as teachers of SRL. Learners increasingly depended on PAs to prompt SRL strategies as time on task progressed. PAs are effective scaffolders of SRL with more research needed to understand their role as teachers of SRL. PA scaffolding is more essential as time on task progresses. When deploying specific cognitive and metacognitive SRL strategies, the relevance of the content to learners' subgoals should be taken into account.
This content will become publicly available on May 31, 2024
Capturing evidence for dynamic changes in self‐regulated learning (SRL) behaviours resulting from interventions is challenging for researchers. In the current study, we identified students who were likely to do poorly in a biology course and those who were likely to do well. Then, we randomly assigned a portion of the students predicted to perform poorly to a science of learning to learn intervention where they were taught SRL study strategies. Learning outcome and log data (257 K events) were collected from
What is already known about this topic Self‐regulated learning (SRL) knowledge and skills are strong predictors of postsecondary STEM student success. SRL is a dynamic, temporal process that leads to purposeful student engagement. Methods and metrics for measuring dynamic SRL behaviours in learning contexts are needed. What this paper adds A Markov process for measuring dynamic SRL processes using log data. Evidence that dynamic, interaction‐dominant aspects of SRL predict student achievement. Evidence that SRL processes can be meaningfully impacted through educational intervention. Implications for theory and practice Complexity approaches inform theory and measurement of dynamic SRL processes. Static representations of dynamic SRL processes are promising learning analytics metrics. Engineered features of LMS usage are valuable contributions to AI models.
- Award ID(s):
- 1821601
- NSF-PAR ID:
- 10419812
- Publisher / Repository:
- Wiley-Blackwell
- Date Published:
- Journal Name:
- British Journal of Educational Technology
- Volume:
- 54
- Issue:
- 5
- ISSN:
- 0007-1013
- Page Range / eLocation ID:
- p. 1204-1221
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
more » « less
Abstract N = 118) were randomly assigned to a condition:Control Condition (i.e. learners could only self‐initiate SRL strategies) andPrompt and Feedback Condition (i.e. PAs prompted learners to engage in SRL). Learners' log‐file data captured when strategies were used, the initiator of the strategy (i.e. learner and PA), and the relevance of instructional content pages in relation to learner subgoals. While results showed that PAs were effective scaffolders of SRL in which they prompted learners to engage in SRL strategies more when content was relevant towards their subgoals and as time on page and task increased, there were mixed findings about the effectiveness of PAs as teachers of SRL. Findings show how production rules guiding PA prompts can improve their scaffolding and teaching of SRL across the learning task – through contextualizing SRL strategies to the instructional content and in relation to the relevance of the content to learners' subgoals.Practitioner notes What is already known about this topic What this paper adds Implications for practice and/or policy -
more » « less
Abstract This paper presents an implementation of Connected Spaces (CxS)—an ambient help seeking interface designed and developed for a project‐based computing classroom. We use actor network theory (ANT) to provide an underutilized posthumanist lens to understand the creation of collaborative connections in this Computational Action‐based implementation. Posthumanism offers an emerging and critical extension to sociocultural perspectives on understanding learning, by pushing us to decenter the human, and consider the active roles that human and non‐human entities play in learning environments by actively shaping each other. We analyse how students in this class adjusted their help‐seeking and collaborative habits following the introduction of CxS, a tool designed to foster (more inter‐group) collaboration. ANT proposes generalized symmetry—a principle of considering human, non‐human and more than human entities with equivalent and comparable agency, leading to describing phenomena as networks of actors in different evolving relationships with each other. Analysing collaborative interactions as fostered by CxS using an ANT approach supports design‐based research—an iterative design revision process highlighting understandings about design as well as learning—by providing a temporal and informative lens into the relationship between actors and tools within the environment. Our key findings include a framing of technologies in classrooms as bridging
agentic gaps between students and becoming actors engaging in different behaviours; learners enacting new agencies through technologies (for instance a more comfortable non‐intrusive help seeker), and the need for voicing and teachers to connect help networks in CxS equipped classrooms.Practitioner notes What is already known about this topic
Collaborative learning is a valuable skill and practice; opportunities to mentor others are critical in empowering minoritized learners, especially in STEM and computing disciplines.
School norms solidify a power and expertise hierarchy between teachers and learners and fail to productively support learners in learning from each other.
Additionally, lack of awareness about peers' knowledge is a common hindrance in students knowing who to ask for help and how.
What this paper adds
An example of a designed interface called Connected Spaces with potential to foster more inter‐student collaboration, especially outside of mandated within‐group collaboration—in the form of cross‐group help seeking and help giving.
A design based research study using actor network theory highlighting the limitations of Connected Spaces in sparking notable behaviour change among students by itself but being retooled as a teacher support tool in enabling cross‐group collaborations.
Presenting conceptions of collaboration through technologies as bridging agentic gaps and acting with new agencies in performing help‐seeking related actions.
Provoking the idea of testing emerging technologies in classrooms along with sharing our analyses and reflections with the classroom as a key idea in computing education—surfacing the gap between designed intentions and the different kinds of extra social work needed in the on‐ground success of different technologies.
Implications for practice and/or policy
Designers and researchers should create and test more interfaces alongside teachers across different classrooms and contexts aimed at supporting different kinds of voluntary collaborative interactions.
Curricula, standards and school practices should further center providing students with opportunities to engage as mentors and build communities of learning across disciplines to empower minoritized students.
Researchers engaging in design based research should consider using more posthumanist lenses to examine educational technologies and how they affect change in learning environments.
-
more » « less
Abstract This descriptive study focuses on using voice activity detection (VAD) algorithms to extract student speech data in order to better understand the collaboration of small group work and the impact of teaching assistant (TA) interventions in undergraduate engineering discussion sections. Audio data were recorded from individual students wearing head‐mounted noise‐cancelling microphones. Video data of each student group were manually coded for collaborative behaviours (eg, group task relatedness, group verbal interaction and group talk content) of students and TA–student interactions. The analysis includes information about the turn taking, overall speech duration patterns and amounts of overlapping speech observed both when TAs were intervening with groups and when they were not. We found that TAs very rarely provided explicit support regarding collaboration. Key speech metrics, such as amount of turn overlap and maximum turn duration, revealed important information about the nature of student small group discussions and TA interventions. TA interactions during small group collaboration are complex and require nuanced treatments when considering the design of supportive tools.
Practitioner notes What is already known about this topic
Student turn taking can provide information about the nature of student discussions and collaboration.
Real classroom audio data of small groups typically have lots of background noise and present challenges for audio analysis.
TAs have little training in how to productively intervene with students about collaborative skills.
What this paper adds
TA interaction with groups primarily focused on task progress and understanding of concepts with negligible explicit support on building collaborative skills.
TAs intervened with the groups often which gave the students little time for uptake of their suggestions or deeper discussion.
Student turn overlap was higher without the presence of TAs.
Maximum turn duration can be an important real‐time turn metric to identify the least verbally active student participant in a group.
Implications for practice and/or policy
TA training should include information about how to monitor groups for collaborative behaviours and when and how they should intervene to provide feedback and support.
TA feedback systems should keep track of previous interventions by TAs (especially in contexts where there are multiple TAs facilitating) and the duration since previous intervention to ensure that TAs do not intervene with a group too frequently with little time for student uptake.
Maximum turn duration could be used as a real‐time metric to identify the least verbally active student in a group so that support could be provided to them by the TAs.
-
more » « less
Abstract Much attention in constructionism has focused on designing tools and activities that support learners in designing fully finished and functional applications and artefacts to be shared with others. But helping students learn to debug their applications often takes on a surprisingly more instructionist stance by giving them checklists, teaching them strategies or providing them with test programmes. The idea of designing bugs for learning—or
debugging by design —makes learners agents of their own learning and, more importantly, of making and solving mistakes. In this paper, we report on our implementation of ‘Debugging by Design’ activities in a high school classroom over a period of 8 hours as part of an electronic textiles unit. Students were tasked to craft the electronic textile artefacts with problems or bugs for their peers to solve. Drawing on observations and interviews, we answer the following research questions: (1) How did students participate in making bugs for others? (2) What did students gain from designing and solving bugs for others? In the discussion, we address the opportunities and challenges that designing personally and socially meaningful failure artefacts provides for becoming objects‐to‐think‐with and objects‐to‐share‐with in student learning and promoting new directions in constructionism.Practitioner notes What is already known about this topic
There is substantial evidence for the benefits of learning programming and debugging in the context of constructing personally relevant and complex artefacts, including electronic textiles.
Related, work on productive failure has demonstrated that providing learners with strategically difficult problems (in which they ‘fail’) equips them to better handle subsequent challenges.
What this paper adds
In this paper, we argue that designing bugs or ‘failure artefacts’ is as much a constructionist approach to learning as is designing fully functional artefacts.
We consider how ‘failure artefacts’ can be both objects‐to‐learn‐with and objects‐to‐share‐with.
We introduce the concept of ‘Debugging by Design’ (DbD) as a means to expand application of constructionism to the context of developing ‘failure artifacts’.
Implications for practice and/or policy
We conceptualise a new way to enable and empower students in debugging—by designing creative, multimodal buggy projects for others to solve.
The DbD approach may support students in near‐transfer of debugging and the beginning of a more systematic approach to debugging in later projects and should be explored in other domains beyond e‐textiles.
New studies should explore learning, design and teaching that empower students to design bugs in projects in mischievous and creative ways.
-
more » « less
Abstract This paper describes a Human‐Centred Learning Analytics (HCLA) design approach for developing learning analytics (LA) dashboards for K‐12 classrooms that maintain both contextual relevance and scalability—two goals that are often in competition. Using mixed methods, we collected observational and interview data from teacher partners and assessment data from their students' engagement with the lesson materials. This DBR‐based, human‐centred design process resulted in a dashboard that supported teachers in addressing their students' learning needs. To develop the dashboard features that could support teachers, we found that a design refinement process that drew on the insights of teachers with varying teaching experience, philosophies and teaching contexts strengthened the resulting outcome. The versatile nature of the approach, in terms of student learning outcomes, makes it useful for HCLA design efforts across diverse K‐12 educational contexts.
Practitioner notes What is already known about this topic
Learning analytics that are aligned to both a learning theory and learning design support student learning.
LA dashboards that support users to understand the associated learning analytics data provide actionable insight.
Design‐based research is a promising methodology for Human‐Centred Learning Analytics design, particularly in the K‐12 educational context.
What this paper adds
Leveraging a longstanding, yet fluid, research‐practice partnership is an effective design‐based research adaptation for addressing the high variation in instructional practices that characterize K‐12 education.
Using both quantitative and qualitative data that reflects students' developing knowledge effectively supports teachers' inquiry into student learning.
Teachers' use of learning analytics dashboards is heavily influenced by their perspectives on teaching and learning.
Implications for practice and/or policy
Impact on student learning outcomes, alongside usability and feasibility, should be included as a necessary metric for the effectiveness of LA design.
LA dashboard developers should both leverage learning data that reflect students' developing knowledge and position teachers to take responsive pedagogical action to support student learning.
LA researchers and developers should utilize a long‐term, yet fluid, research‐practice partnership to form a multi‐stakeholder, multidisciplinary design team for Human‐Centred Learning Analytics design.
