More Like this

1. A spatiotemporal analysis of teacher practices in supporting student learning and engagement in an AI-enabled classroom

   https://doi.org/10.1007/978-3-031-36272-9_37  

   Karumbaiah, S. ; Borchers, C. ; Falhs, A. C. ; Holstein, K. ; Rummel, N. ; Aleven, V. ( July 2023 , Springer)

   Wang, N. ; Rebolledo-Mendez, G. ; Matsuda, N. ; Santos, O.C. ; Dimitrova, V. (Ed.)

   Research indicates that teachers play an active and important role in classrooms with AI tutors. Yet, our scientific understanding of the way teacher practices around AI tutors mediate student learning is far from complete. In this paper, we investigate spatiotemporal factors of student-teacher interactions by analyzing student engagement and learning with an AI tutor ahead of teacher visits (defined as episodes of a teacher being in close physical proximity to a student) and immediately following teacher visits. To conduct such integrated, temporal analysis around the moments when teachers visit students, we collect fine-grained, time-synchronized data on teacher positions in the physical classroom and student interactions with the AI tutor. Our case study in a K12 math classroom with a veteran math teacher provides some indications on factors that might affect a teacher’s decision to allocate their limited classroom time to their students and what effects these interactions have on students. For instance, teacher visits were associated more with students’ in-the-moment behavioral indicators (e.g., idleness) than a broader, static measure of student needs such as low prior knowledge. While teacher visits were often associated with positive changes in student behavior afterward (e.g., decreased idleness), there could be a potential mismatch between students visited by the teacher and who may have needed it more at that time (e.g., students who were disengaged for much longer). Overall, our findings indicate that teacher visits may yield immediate benefits for students but also that it is challenging for teachers to meet all needs - suggesting the need for better tool support. 

   more » « less
2. Designing for complementarity: Teacher and student needs for orchestration support in AI-enhanced classrooms

   https://doi.org/10.1007/978-3-030-23204-7_14  

   Holstein, Kenneth ; McLaren, Bruce M. ; Aleven, Vincent ( June 2019 , Proceedings, 20th International Conference on Artificial Intelligence in Education, AIED 2019)

   As artificial intelligence (AI) increasingly enters K-12 classrooms, what do teachers and students see as the roles of human versus AI instruction, and how might educational AI (AIED) systems best be designed to support these complementary roles? We explore these questions through participatory design and needs validation studies with K-12 teachers and students. Using human-centered design methods rarely employed in AIED research, this work builds on prior findings to contribute: (1) an analysis of teacher and student feedback on 24 design concepts for systems that integrate human and AI instruction; and (2) participatory speed dating (PSD): a new variant of the speed dating design method, involving iterative concept generation and evaluation with multiple stakeholders. Using PSD, we found that teachers desire greater real-time support from AI tutors in identifying when students need human help, in evaluating the impacts of their own help-giving, and in managing student motivation. Meanwhile, students desire better mechanisms to signal help-need during class without losing face to peers, to receive emotional support from human rather than AI tutors, and to have greater agency over how their personal analytics are used. This work provides tools and insights to guide the design of more effective human–AI partnerships for K-12 education. 

   more » « less
3. Teacher noticing and student learning in human-AI partnered classrooms: A multimodal analysis

   Karumbaiah, S. ; Borchers, C. ; Falhs, A. ; Holstein, K. ; Rummel, N. ; Aleven, V. ( June 2023 , International Society of the Learning Sciences)

   Blikstein, P. ; Van Aalst, J. ; Kizito, R. ; Brennan, K. (Ed.)

   Past research shows that teacher noticing matters for student learning, but little is known about the effects of AI-based tools designed to augment teachers’ attention and sensemaking. In this paper, we investigate three multimodal measures of teacher noticing (i.e., gaze, deep dive into learning analytics in a teacher tool, and visits to individual students), gleaned from a mixed reality teacher awareness tool across ten classrooms. Our analysis suggests that of the three noticing measures, deep dive exhibited the largest association with learning gains when adjusting for students’ prior knowledge and tutor interactions. This finding may indicate that teachers identified students most in need based on the deep dive analytics and offered them support. We discuss how these multimodal measures can make the constraints and effects of teacher noticing in human-AI partnered classrooms visible. 

   more » « less
4. Co‐designing teacher support technology for problem‐based learning in middle school science

   https://doi.org/10.1111/bjet.13363  

   Hutchins, Nicole M. ; Biswas, Gautam ( July 2023 , British Journal of Educational Technology)

   This paper provides an experience report on a co‐design approach with teachers to co‐create learning analytics‐based technology to support problem‐based learning in middle school science classrooms. We have mapped out a workflow for such applications and developed design narratives to investigate the implementation, modifications and temporal roles of the participants in the design process. Our results provide precedent knowledge on co‐designing with experienced and novice teachers and co‐constructing actionable insight that can help teachers engage more effectively with their students' learning and problem‐solving processes during classroom PBL implementations.

   What is already known about this topic

   Success of educational technology depends in large part on the technology's alignment with teachers' goals for their students, teaching strategies and classroom context.

   Teacher and researcher co‐design of educational technology and supporting curricula has proven to be an effective way for integrating teacher insight and supporting their implementation needs.

   Co‐designing learning analytics and support technologies with teachers is difficult due to differences in design and development goals, workplace norms, and AI‐literacy and learning analytics background of teachers.

   What this paper adds

   We provide a co‐design workflow for middle school teachers that centres on co‐designing and developing actionable insights to support problem‐based learning (PBL) by systematic development of responsive teaching practices using AI‐generated learning analytics.

   We adapt established human‐computer interaction (HCI) methods to tackle the complex task of classroom PBL implementation, working with experienced and novice teachers to create a learning analytics dashboard for a PBL curriculum.

   We demonstrate researcher and teacher roles and needs in ensuring co‐design collaboration and the co‐construction of actionable insight to support middle school PBL.

   Implications for practice and/or policy

   Learning analytics researchers will be able to use the workflow as a tool to support their PBL co‐design processes.

   Learning analytics researchers will be able to apply adapted HCI methods for effective co‐design processes.

   Co‐design teams will be able to pre‐emptively prepare for the difficulties and needs of teachers when integrating middle school teacher feedback during the co‐design process in support of PBL technologies.

    

   more » « less
5. Intelligent teacher dashboards that support students’ self-regulated learning, engagement, and teachers’ decision making

   https://doi.org/https://doi.org/10.3389/feduc.2021.570229  

   Wiedbusch, M. ( April 2021 , Frontiers in education)

   Michalsky, Tova ; Moos, Daniel (Ed.)

   Teachers’ ability to self-regulate their own learning is closely related to their competency to enhance self-regulated learning (SRL) in their students. Accordingly, there is emerging research for the design of teacher dashboards that empower instructors by providing access to quantifiable evidence of student performance and SRL processes. Typically, they capture evidence of student learning and performance to be visualized through activity traces (e.g., bar charts showing correct and incorrect response rates, etc.) and SRL data (e.g., eye-tracking on content, log files capturing feature selection, etc.) in order to provide teachers with monitoring and instructional tools. Critics of the current research on dashboards used in conjunction with advanced learning technologies (ALTs) such as simulations, intelligent tutoring systems, and serious games, argue that the state of the field is immature and has 1) focused only on exploratory or proof-of-concept projects, 2) investigated data visualizations of performance metrics or simplistic learning behaviors, and 3) neglected most theoretical aspects of SRL including teachers’ general lack of understanding their’s students’ SRL. Additionally, the work is mostly anecdotal, lacks methodological rigor, and does not collect critical process data (e.g. frequency, duration, timing, or fluctuations of cognitive, affective, metacognitive, and motivational (CAMM) SRL processes) during learning with ALTs used in the classroom. No known research in the areas of learning analytics, teacher dashboards, or teachers’ perceptions of students’ SRL and CAMM engagement has systematically and simultaneously examined the deployment, temporal unfolding, regulation, and impact of all these key processes during complex learning. In this manuscript, we 1) review the current state of ALTs designed using SRL theoretical frameworks and the current state of teacher dashboard design and research, 2) report the important design features and elements within intelligent dashboards that provide teachers with real-time data visualizations of their students’ SRL processes and engagement while using ALTs in classrooms, as revealed from the analysis of surveys and focus groups with teachers, and 3) propose a conceptual system design for integrating reinforcement learning into a teacher dashboard to help guide the utilization of multimodal data collected on students’ and teachers’ CAMM SRL processes during complex learning. 

   more » « less