skip to main content


Title: Agent-in-the-Loop: Conversational Agent Support in Service of Reflection for Learning During Collaborative Programming
Dynamic conversational agent-based support for collaborative learning has shown significant positive effects on learning over no-support or static-support control conditions in prior studies. In order to understand the boundary between human-led and AI-led support for collaboration, we compare in this study an approach where the agent’s primary role is to help students regulate their own collaboration with two more typical prompting strategies that are used only during a reflection phase: one designed to provide a specific informational focus for the reflection, and the other designed to draw out evaluation, elaboration, and exploration of alternative perspectives. Significant positive effects on learning over and above just the human-led form of support are observed when either of the prompting strategies are used.  more » « less
Award ID(s):
1917955 1822831
NSF-PAR ID:
10174692
Author(s) / Creator(s):
; ; ; ; ; ; ; ;
Date Published:
Journal Name:
Artificial Intelligence in Education. AIED 2020. Lecture Notes in Computer Science
Volume:
12164
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Several studies have highlighted the positive effects that active learning may have on student engagement and performance. However, the influence of active learning strategies is mediated by several factors, including the nature of the learning environment and the cognitive level of in-class tasks. These factors can affect different dimensions of student engagement such as the nature of social processing in student groups, how knowledge is used and elaborated upon by students during in-class tasks, and the amount of student participation in group activities. In this study involving four universities in the US, we explored the association between these different dimensions of student engagement and the cognitive level of assigned tasks in five distinct general chemistry learning environments where students were engaged in group activities in diverse ways. Our analysis revealed a significant association between task level and student engagement. Retrieval tasks often led to a significantly higher number of instances of no interaction between students and individualistic work, and a lower number of knowledge construction and collaborative episodes with full student participation. Analysis tasks, on the other hand, were significantly linked to more instances of knowledge construction and collaboration with full group participation. Tasks at the comprehension level were distinctive in their association with more instances of knowledge application and multiple types of social processing. The results of our study suggest that other factors such as the nature of the curriculum, task timing, and class setting may also affect student engagement during group work. 
    more » « less
  2. Electrical and computer engineering technologies have evolved into dynamic, complex systems that profoundly change the world we live in. Designing these systems requires not only technical knowledge and skills but also new ways of thinking and the development of social, professional and ethical responsibility. A large electrical and computer engineering department at a Midwestern public university is transforming to a more agile, less traditional organization to better respond to student, industry and society needs. This is being done through new structures for faculty collaboration and facilitated through departmental change processes. Ironically, an impetus behind this effort was a failed attempt at department-wide curricular reform. This failure led to the recognition of the need for more systemic change, and a project emerged from over two years of efforts. The project uses a cross-functional, collaborative instructional model for course design and professional formation, called X-teams. X-teams are reshaping the core technical ECE curricula in the sophomore and junior years through pedagogical approaches that (a) promote design thinking, systems thinking, professional skills such as leadership, and inclusion; (b) contextualize course concepts; and (c) stimulate creative, socio-technical-minded development of ECE technologies. An X-team is comprised of ECE faculty members including the primary instructor, an engineering education and/or design faculty member, an industry practitioner, context experts, instructional specialists (as needed to support the process of teaching, including effective inquiry and inclusive teaching) and student teaching assistants. X-teams use an iterative design thinking process and reflection to explore pedagogical strategies. X-teams are also serving as change agents for the rest of the department through communities of practice referred to as Y-circles. Y-circles, comprised of X-team members, faculty, staff, and students, engage in a process of discovery and inquiry to bridge the engineering education research-to-practice gap. Research studies are being conducted to answer questions to understand (1) how educators involved in X-teams use design thinking to create new pedagogical solutions; (2) how the middle years affect student professional ECE identity development as design thinkers; (3) how ECE students overcome barriers, make choices, and persist along their educational and career paths; and (4) the effects of department structures, policies, and procedures on faculty attitudes, motivation and actions. This paper will present the efforts that led up to the project, including failures and opportunities. It will summarize the project, describe related work, and present early progress implementing new approaches. 
    more » « less
  3. null (Ed.)
    Through school-university partnerships that situate learning within culturally relevant educational experiences, faculty, preservice teachers, and school-based educators are able to co-construct and share scientific knowledge. This knowledge consists of pedagogical content knowledge and funds of knowledge that include both knowledge and skills developed in cultural context that have evolved historically. In early childhood education, culturally relevant Science, Technology, Engineering, Arts, and Mathematics (STEAM) learning experiences are particularly important for young children's cognitive and social emotional development. This paper describes how intentional co-planning and collaboration to celebrate the US Read across America Day provided over 100 preschool children in eight classrooms with access to STEAM lessons virtually led by university preservice teachers in partnership with educators in the school. These activities engaged children in exploring art, computer science, physical science, engineering, and mathematics within the context of a culturally relevant version of the fairy tale Goldilocks and the Three Bears. Lessons implemented as part of school-university partnerships support Black and Latinx children's development of a sense of belonging in STEAM. Further, these experiences enhance teacher candidates' abilities to engage in culturally responsive STEAM teaching while receiving ongoing guidance and education from university faculty and school-based educators. Teacher education programs within higher education institutions should embrace school- university partnerships as contexts for the development of shared scientific knowledge and discourse since the benefits are twofold. First, children and teachers gain access to, and engage with, innovative STEAM experiences. Second, preservice teachers learn culturally relevant research-based instructional strategies through university coursework situated in authentic learning experiences; thus, their learning as teacher candidates is enhanced through planning, implementation, evaluation, and critical reflection. 
    more » « less
  4. Evidence has shown that facilitating student-centered learning (SCL) in STEM classrooms enhances student learning and satisfaction [1]–[3]. However, despite increased support from educational and government bodies to incorporate SCL practices [1], minimal changes have been made in undergraduate STEM curriculum [4]. Faculty often teach as they were taught, relying heavily on traditional lecture-based teaching to disseminate knowledge [4]. Though some faculty express the desire to improve their teaching strategies, they feel limited by a lack of time, training, and incentives [4], [5]. To maximize student learning while minimizing instructor effort to change content, courses can be designed to incorporate simpler, less time-consuming SCL strategies that still have a positive impact on student experience. In this paper, we present one example of utilizing a variety of simple SCL strategies throughout the design and implementation of a 4-week long module. This module focused on introductory tissue engineering concepts and was designed to help students learn foundational knowledge within the field as well as develop critical technical skills. Further, the module sought to develop important professional skills such as problem-solving, teamwork, and communication. During module design and implementation, evidence-based SCL teaching strategies were applied to ensure students developed important knowledge and skills within the short timeframe. Lectures featured discussion-based active learning exercises to encourage student engagement and peer collaboration [6]–[8]. The module was designed using a situated perspective, acknowledging that knowing is inseparable from doing [9], and therefore each week, the material taught in the two lecture sessions was directly applied to that week’s lab to reinforce students’ conceptual knowledge through hands-on activities and experimental outcomes. Additionally, the majority of assignments served as formative assessments to motivate student performance while providing instructors with feedback to identify misconceptions and make real-time module improvements [10]–[12]. Students anonymously responded to pre- and post-module surveys, which focused on topics such as student motivation for enrolling in the module, module expectations, and prior experience. Students were also surveyed for student satisfaction, learning gains, and graduate student teaching team (GSTT) performance. Data suggests a high level of student satisfaction, as most students’ expectations were met, and often exceeded. Students reported developing a deeper understanding of the field of tissue engineering and learning many of the targeted basic lab skills. In addition to hands-on skills, students gained confidence to participate in research and an appreciation for interacting with and learning from peers. Finally, responses with respect to GSTT performance indicated a perceived emphasis on a learner-centered and knowledge/community-centered approaches over assessment-centeredness [13]. Overall, student feedback indicated that SCL teaching strategies can enhance student learning outcomes and experience, even over the short timeframe of this module. Student recommendations for module improvement focused primarily on modifying the lecture content and laboratory component of the module, and not on changing the teaching strategies employed. The success of this module exemplifies how instructors can implement similar strategies to increase student engagement and encourage in-depth discussions without drastically increasing instructor effort to re-format course content. Introduction. 
    more » « less
  5. This research explores a novel human-in-the-loop approach that goes beyond traditional prompt engineering approaches to harness Large Language Models (LLMs) with chain-of-thought prompting for grading middle school students’ short answer formative assessments in science and generating useful feedback. While recent efforts have successfully applied LLMs and generative AI to automatically grade assignments in secondary classrooms, the focus has primarily been on providing scores for mathematical and programming problems with little work targeting the generation of actionable insight from the student responses. This paper addresses these limitations by exploring a human-in-the-loop approach to make the process more intuitive and more effective. By incorporating the expertise of educators, this approach seeks to bridge the gap between automated assessment and meaningful educational support in the context of science education for middle school students. We have conducted a preliminary user study, which suggests that (1) co-created models improve the performance of formative feedback generation, and (2) educator insight can be integrated at multiple steps in the process to inform what goes into the model and what comes out. Our findings suggest that in-context learning and human-in-the-loop approaches may provide a scalable approach to automated grading, where the performance of the automated LLM-based grader continually improves over time, while also providing actionable feedback that can support students’ open-ended science learning. 
    more » « less