skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: (2025). Talking to learn: A SoTL study of generative AI-facilitated Feynman reviews. Proceedings of the Artificial Intelligence in Measurement Education Conference.
Structured Generative AI interactions have potential for scaffolding learning. This Scholarship of Teaching and Learning study analyzes 16 undergraduate students’ Feynman-style AI interactions (N=154) across a semester. Qualitative coding of the interactions shows mostly low-level student responses, but some evidence that prompt structure may can promote higher- level cognitive engagement. Results show GAI provides metacognitive support, and suggest the potential of GAI-supported Feynman reviews to provide interactive, personalized learning experiences that align with theories of cognitive engagement and metacognitive support for learning.  more » « less
Award ID(s):
2145478
PAR ID:
10667081
Author(s) / Creator(s):
; ;
Editor(s):
Wilson, J; Ormerod, C; Beiting_Parrish, M
Publisher / Repository:
National Center for Measurement in Education (NCME)
Date Published:
Format(s):
Medium: X
Location:
https://aclanthology.org/2025.aimecon-wip.14/
Sponsoring Org:
National Science Foundation
More Like this
  1. Wilson, J; Ormerod, C; Beiting_Parrish, M (Ed.)
    Structured Generative AI interactions have potential for scaffolding learning. This Scholarship of Teaching and Learning study analyzes 16 undergraduate students’ Feynman-style AI interactions (N=154) across a semester. Qualitative coding of the interactions shows mostly low-level student responses, but some evidence that prompt structure may can promote higherlevel cognitive engagement. Results show GAI provides metacognitive support, and suggest the potential of GAI-supported Feynman reviews to provide interactive, personalized learning experiences that align with theories of cognitive engagement and metacognitive support for learning. 
    more » « less
  2. Despite the potential of generative AI (GenAI) design tools to enhance design processes, professionals often struggle to integrate AI into their workflows. Fundamental cognitive challenges include the need to specify all design criteria as distinct parameters upfront (intent formulation) and designers' reduced cognitive involvement in the design process due to cognitive offloading, which can lead to insufficient problem exploration, underspecification, and limited ability to evaluate outcomes. Motivated by these challenges, we envision novel metacognitive support agents that assist designers in working more reflectively with GenAI. To explore this vision, we conducted exploratory prototyping through a Wizard of Oz elicitation study with 20 mechanical designers probing multiple metacognitive support strategies. We found that agent-supported users created more feasible designs than non-supported users, with differing impacts between support strategies. Based on these findings, we discuss opportunities and tradeoffs of metacognitive support agents and considerations for future AI-based design tools. 
    more » « less
  3. Generative Artificial Intelligence (GAI) has emerged in recent years as an innovative tool with promising potential for enhancing student learning across a broad spectrum of academic disciplines. GAI not only offers students personalized and adaptive learning experiences, but it is also playing an increasingly important role in various industries. As technologies evolve and society adapts to the growing AI revolution, it becomes necessary to train students of all disciplines to become proficient in using GAI. This work builds on studies that have established the effectiveness of intelligent tutoring systems, adaptive learning environments, and the use of virtual reality in education. This work-in-progress paper presents preliminary findings related to the relationship between university students’ area of study and the frequency at which they utilize GAI to aid their learning. Data for this study were collected using a survey distributed to students from eight different colleges at a large Western university as part of a larger ongoing project geared towards gaining insight into student perceptions and use of GAI in higher education. The goal of the overall project is to establish a foundational understanding of how disruptive technologies, like GAI, can promote learner agency. By exploring why and how students choose to engage with these technologies, the project seeks to find proactive approaches to integrate GAI technology into education, ultimately enhancing teaching and learning practices across various disciplines. 
    more » « less
  4. This work-in-progress paper explores university students’ perspectives on Generative Artificial Intelligence (GAI) tools, such as ChatGPT, an increasingly prominent topic in the academic community. There is ongoing debate about whether faculty should teach students how to use GAI tools, restrict their usage to maintain academic integrity, or establish regulatory guidelines for sustained integration into higher education. Unfortunately, limited research exists beyond surface-level policies and educator opinions regarding GAI, and its full impact on student learning remains largely unknown. Therefore, understanding students' perceptions and how they use GAI is crucial to ensuring its effective and ethical integration into higher education. As GAI continues to disrupt traditional educational paradigms, this study seeks to explore how students perceive its influence on their learning and problem-solving. As part of a larger mixed-methods study, this work-in-progress paper presents preliminary findings from the qualitative portion using a phenomenological approach that answers the research question: How do university students perceive disruptive technologies like ChatGPT affecting their education and learning? By exploring the implications of Artificial Intelligence (AI) tools on student learning, academic integrity, individual beliefs, and community norms, this study contributes to the broader discourse on the role of emerging technologies in shaping the future of teaching and learning in education. 
    more » « less
  5. Debugging process plays a crucial role in helping students pinpoint their specific learning weaknesses, allowing them to modify their strategies for enhanced academic performance. Notably, changes in pupil dilation serve as an indicator of arousal associated with confronting learning challenges. This physiological response acts as a “physiological footprint” that reflects cognitive engagement, facilitating internally focused cognitive processes such as insight generation and mind-wandering. In this study, we proposed that pupil dilation could be a valuable predictor of students’ metacognitive awareness throughout the debugging process, specifically within an augmented reality (AR) learning environment. The findings revealed significant differences in pupil dilation among students categorized by their varying levels of debugging, which represents a specific dimension of the Metacognitive Awareness Inventory. 
    more » « less