Although there is growing interest in the use of GAI in education, there is a great need for more empirical studies that discuss its adoption and impact (Farrokhnia et al., 2024). In particular, systematic evaluations of the effectiveness of GAI-based classroom activities remain scarce. Such evaluations should carefully consider factors like pedagogical formats, audience demographics, and disciplinary differences. To help address this gap, we present the findings from a year-long experiment conducted in 2023, where the authors incorporated ChatGPT into college classroom teaching across various subjects and formats, involving both undergraduate and graduate students. This teaching experiment spans two semesters and provides a unique perspective on the dynamics of using ChatGPT in real-world educational settings. Participating students and teachers provided their feedback and reflections upon completion of the teaching experiment, offering valuable insights and practical suggestions for educators interested in using GAI tools in their future teaching practices.
In summary, our goal with this manuscript is to contribute to the growing body of knowledge on implementing GAI in higher education and offer data-driven insights and actionable recommendations for educators and institutions. The structure of this paper is as follows: a review of the relevant literature, a detailed description of the experimental design and methodology, a summary of the experiment findings, and a discussion of the implications and future recommendations.
Driven by significant advancements in machine learning and deep learning, natural language processing and GAI have evolved rapidly over the past decade. One of the most notable recent innovations is OpenAI's ChatGPT (OpenAI, 2022), which was built based on the powerful Generative Pre-trained Transformer (GPT) model capable of producing human-like text and engaging in sophisticated "conversations" with human users. In education contexts, ChatGPT is a novel yet promising tool that offers numerous benefits, including (and not limited to) personalized learning experiences, enhanced engagement, immediate feedback on assignments, easy access to rich information and resources, the ability to facilitate discussions and debates, the promotion of independent learning, and the potential to reduce workload by automating routine tasks. These advantages could contribute to a more dynamic, interactive, and efficient educational environment, ultimately enhancing the overall learning experience for students and teaching experience for instructors.
Despite its potential benefits, the use of GAI in educational contexts also raises concerns, including (among others) the potential to: engender an over-reliance on GAI, reduce development of critical thinking skills, and present ethical issues related to data transparency and accessibility. A developing GAI-related debate among educators reflects a broader discussion on balancing the innovative potential of GAI with the need to maintain rigorous educational standards (e.g., Yu, 2023). While some view GAI as a transformative tool that can enrich the learning experience, others fear this type of technology may diminish the quality of education by fostering dependence and undermining academic integrity.
Numerous studies have explored this topic since the advent of ChatGPT. To provide a general overview of the current state of GAI research within the educational sector, particularly in the traditional schooling systems, we conducted an extensive literature search covering studies published up to April 2024. Given the rapid pace of AI advancements, newer studies may have emerged since our search; however, we believe our analysis captures key trends and patterns that remain relevant and consistent over the early stage of GAI adoption in education. Our focus was on peerreviewed academic journal articles that specifically examined the use of ChatGPT as a tool in education. Using Google Scholar and Web of Science, we filtered articles to include only those with "education" and "ChatGPT" in the title or abstract. We then coded each article as either empirical (E) or review (R) in form, and whether it focused on education within schools (1) or not (2). This search resulted in 376 articles, 271 of which directly addressed the use of ChatGPT in school settings. The remaining studies primarily focused on ChatGPT's application in various professional fields, such as healthcare, research, electronics, white-collar work, language, and marketing, with healthcare being the predominant area. Notably, a large number of review articles (n = 151) were identified during the search and generally explored the broader implications of ChatGPT in the context of education and training. These articles frequently discussed both the potential benefits and risks of using ChatGPT, with most adopting a neutral or cautiously optimistic stance. This cautious approach is understandable, as ChatGPT is still in its early stages of development, and its longterm impacts have yet to be established through more robust empirical research.
For the present study, we focus on empirical studies exclusively within the education system. Through the literature search process described above, we identified 158 such studies for further analysis. A summary table of these studies is presented in Table 1, which shows that most of the existing research in this domain has been done in a higher, post-secondary education context. The number of studies that focus on graduate-level (n = 64) and undergraduatelevel (n = 70) topics are similar, and collectively account for 85% of the total studies identified. In contrast, only 16 studies focus on K-12 grade levels, while eight studies do not clearly specify the educational level context information.
Among the studies identified, the most common category (n = 40) focuses on testing ChatGPT's proficiency and utility for generating and responding to exam and assignment questions. These studies primarily evaluate ChatGPT's performance on exam and assignment questions relevant to higher education classroom content (e.g., Currie and Barry, 2023). In particular, a large portion of studies within the category use medical licensing exam questions to assess ChatGPT's capabilities (e.g., Wang et al., 2023).
The second most prevalent category evident in these identified studies (n = 36) explores how ChatGPT can support both teachers and students as an innovative tool. While some studies focus on using ChatGPT to develop educational content and design learning tools (e.g., Jeon and Lee, 2023), others investigate its potential as a teaching assistant to aid student learning (e.g., Lee et al., 2024). The third category (n = 30) examines student and teacher perceptions of ChatGPT, often assessing its perceived usefulness and potential risks (e.g., Zou and Huang, 2023). Although fewer in number, several studies investigate the current prevalence of ChatGPT use by students and its impact on student
Guo et al. 10.3389/feduc.2025.1574477 TABLE 1 Empirical studies of use of ChatGPT in education. Education level Topics Graduate-level Undergraduate K-12 Unspecified Tools to Support Teachers/students Prevalence of use among students Cross et al. (2023) Crcek and Patekar (2023) Duong et al. (2023) Polyportis (2024) Salifu et al. (2024) Raman et al. (2023) --(Continued) Frontiers in Education 03 frontiersin.org Guo et al. 10.3389/feduc.2025.1574477 TABLE 1 (Continued) Education level Topics Graduate-level Undergraduate K-12 Unspecified Guo et al. 10.3389/feduc.2025.1574477
performance. These studies are mostly at the undergraduate level, with some examining K-12 educational contexts. Surprisingly, there are not many studies directly comparing ChatGPT and student responses to various homework assignments (with few attempts such as in Currie and Barry, 2023), which is a significant concern for educators regarding plagiarism. Finally, in the "Others" category shown in Table 1, an emerging trend in the literature is the use of ChatGPT to support research processes, particularly in manuscript generation at the graduate education level.
From our review, we noticed that while existing research offers some, mostly positive signs and insights regarding the use of ChatGPT, additional empirical studies are needed to promote the responsible and effective use of GAI in classrooms and other learning settings. Unlike many existing studies that capture perceptions at a single point in time, the present year-long experiment follows a consistent framework across two consecutive university semesters. This approach allows us to track changes in learning effectiveness and provide a more comprehensive view of student perceptions. By assessing how student perspectives have evolved with the rising popularity and understanding of GAI's benefits and limitations, our study offers deeper insights into current trends in student perceptions within higher education, as well as students' adaptation to this new technology.
Moreover, the present study connects student perceptions directly to various class activities in different academic disciplines (e.g., a GAI-facilitated coding activity in statistics courses in psychology and a GAI-enhanced discussion activity in physics), rather than gathering general opinions about non-contextualized GAI applications. This method yields more refined and meaningful data on students' perceived usefulness of the activities themselves, providing valuable information for researchers and educators interested in adopting similar classroom instruction methods. Further, direct comparisons across distinct activity designs and over time offer invaluable opportunities for teachers to reflect on the effective integration of GAI into their own teaching practices.
In the following section, we explain the details of our class activities incorporating ChatGPT into classroom settings during the spring and fall semesters of 2023.
2 Experimenting with the integration of ChatGPT in classroom activities
The primary task of this classroom activity was to facilitate students' familiarity with Python programming, with a specific goal of building and improving models for handwritten digit recognition. Throughout this activity, students were guided in constructing a neural network model to recognize handwritten digits. The learning process for this activity involved students executing instructor-provided Python code examples, responding to various related queries, and iteratively modifying their code to enhance model performance. Students were introduced to ChatGPT as a potentially helpful resource and were permitted to access and utilize ChatGPT to find answers to their questions. This exercise was designed to foster an interactive, hands-on approach to learning Python and understanding the process of building a neural network model. The length of this class activity was one hour. The implementation details of this class activity are outlined below (Also see Figure 1 for workflow illustration for all class activities):
First, the activity task and goal were explained to students. The students were then introduced to Google Colab and ChatGPT, and guided to a level of familiarity with these tools. Next, after ensuring that all students had access to both tools and were comfortable with their basic operations, each student was provided with a copy of an interactive Python notebook. This notebook contained pre-written Python code snippets and accompanying questions. An excerpt from the notebook is provided in Figure 2, while the complete notebook can be found in the Supplementary materials.
Next, students were grouped into pairs and collaboratively tackled a series of problems within a shared notebook. The exercise was structured around four sets of questions that corresponded to four distinct code blocks provided with the activity materials. To maintain a cohesive learning pace and ensure no one was left behind, the students were asked to address the questions sequentially and as a class. Students were also encouraged to ask the instructor any questions during the problem-solving process, fostering an open dialogue within the classroom. Before progressing to the next problem, the instructor reviewed and explained each previous question.
It is important to note that the teacher actively monitored the class to ensure effective communication between the students and ChatGPT. While the teacher did not prescribe a onesize-fits-all "prompt, " guidance and suggestions regarding the selection of prompts were provided, especially when students encountered difficulties.
After the class activity, students were given a brief survey (summarized in Table 2) regarding their general reactions or perceptions toward the use of ChatGPT in the activity, along with some basic demographic information (e.g., gender, race, and first-generation college status).
The primary goal of this activity was for students to grasp the physical meaning of "phase" in wave mechanics and its role in generating interference and diffraction patterns in acoustics and optics. In physics, phase is a key parameter that influences the behavior and interaction of waves, particles, and various physical systems. Understanding phase is crucial for describing a wide range of physical processes, including wave interference, signal processing, synchronization, and the dynamics of electrical circuits and mechanical vibrations. In thermodynamics and condensed matter physics, phase also refers to different states of matter and their transitions, underscoring its broad relevance across multiple domains in physics. This one-hour class activity was specifically designed for non-physics STEM undergraduates. The implementation details of this class activity are outlined below:
The activity started with the instructor's brief introduction to wave properties, including wavelength, frequency, amplitude, Workflow illustration of each class activity. and phase. Phase was explained as a wave's position within its cycle, a concept that can be observed in everyday life, where its critical role in understanding phenomena such as interference and diffraction was emphasized.
Next, students were divided into small groups and tasked with exploring different aspects of phase using ChatGPT.foot_2 Each group was assigned a specific question, such as "How do phase shifts affect interference patterns?" or "What is the difference between constructive and destructive interference?" By interacting with ChatGPT, students asked a series of individual questions and received tailored explanations. They also took notes on the information provided by ChatGPT and discussed their findings within their groups.
Following their interactions with ChatGPT, each group presented their findings to the class, where students explained the concept of phase and its effects on wave behavior based on their AIassisted research. The instructor summarized the key points on the whiteboard and clarified any misconceptions. To deepen students' understanding, the instructor facilitated a brief question-andanswer discussion where students could ask additional questions and seek further clarification. This discussion helped resolve any remaining uncertainties and reinforced the main concepts. The instructor concluded the activity by having students reflect on how using ChatGPT had enhanced their understanding of phase. The students were instructed to respond to the same brief survey described in the in-class activity above.
Over the course of this year-long experiment (i.e., spring and fall semesters of 2023), a total of 169 students participated in one of the class experiments just described, providing their feedback in response to the student survey. The survey items and basic descriptive statistics of students' responses are presented in Table 2. In the psychology discipline, both graduate and undergraduate students from multiple courses (i.e., undergraduate and graduate statistics courses) participated in the coding activity experiment. A total of 91 students provided feedback, with 47% being graduate students, 79% female, 78% White, and 34% first-generation college students. For physics, 78 undergraduate students participated (all enrolled in a 1,000-level general education course focused on fundamentals of electromagnetism and optics). Among these students, 64% were female, 88% White, and 23% first-generation college students.
Overall, students found ChatGPT easy to use and beneficial as a resource for learning. They enjoyed interacting with the tool, felt it boosted their confidence and motivation, and saw it as a valuable classroom supplement. They also believed it provided fair educational opportunities and expressed a desire to continue using ChatGPT in the future. At the same time, the students did not believe that ChatGPT could replace the instructors in classrooms to address their individual needs. Additionally, they expressed varying degrees of concern over the use of ChatGPT.
Next, we present findings that compare students' perceptions across different subgroups of interest. All the analysis details can be found in Table 3. The comparative analysis first reveals that
Guo et al. 10.3389/feduc.2025.1574477 TABLE 2 Basic descriptive statistics for students' survey data. Survey item (Likert scale 1-5) All data (n = 169) PHYS PSYC M (SD) T1 M (n = 42) T2 M (n = 36) Combined M T1 M (n = 30) T2 M (n = 61) 1. ChatGPT is easy to use 4.08 (0.90) 3.67 4.11 3.87 4.37 4.20 4.25 2. ChatGPT seems to be useful for my learning 3.78 (0.98) 3.26 3.58 3.41 4.10 4.10 4.10 3. Use of ChatGPT could improve my learning 3.82 (0.95) 3.33 3.72 3.51 3.83 4.20 4.08 4. I have fun interacting with ChatGPT 3.78 (0.98) 3.38 3.72 3.54 4.23 3.85 3.98 5. The use of ChatGPT helps me connect ideas in new ways 3.67 (0.96) 3.31 3.53 3.41 3.80 3.95 3.90 6. The use of ChatGPT can help me develop confidence in the subject area 3.70 (1.00) 3.29 3.69 3.47 3.90 3.90 3.90 7. The use of ChatGPT motivated me to learn certain new skills 3.35 (1.00) 2.93 3.22 3.06 3.63 3.57 3.59 8. Use of ChatGPT can be important supplement to the classroom 3.65 (1.04) 3.19 3.67 3.41 3.70 3.93 3.86 9. I think the use of ChatGPT in classroom is a good idea. 3.45 (1.03) 3.00 3.31 3.14 3.47 3.84 3.71 10. I think ChatGPT provides a fair education opportunity to all the students 3.43 (1.13) 2.95 3.53 3.22 3.20 3.80 3.60 11. I plan to or will continue to use ChatGPT in the future 3.60 (1.19) 2.93 3.50 3.19 3.90 3.97 3.95 12. I am satisfied with the ChatGPT activity. 3.62 (0.99) 3.19 3.72 3.44 3.73 3.80 3.78 13. I think ChatGPT can better address my individual questions than teachers 2.64 (1.12) 2.76 2.44 2.62 2.57 2.72 2.67 14. I have some concerns over the use of ChatGPT in classrooms 3.36 (1.15) 3.69 3.28 3.50 3.40 3.16 3.24 15. Using ChatGPT takes too much of my time 2.11 (0.82) 2.43 2.11 2.28 1.87 2.00 1.96 16. I am not comfortable with the idea of use of ChatGPT 2.47 (1.23) 2.95 2.33 2.67 2.37 2.26 2.30 T1 refers to the Spring semester of 2023, and T2 refers to the Fall semester of 2023. Frontiers in Education 08 frontiersin.org Guo et al. 10.3389/feduc.2025.1574477 TABLE 3 Students' perception comparison among various subgroups. Survey item PHYS vs. PSYC (PHYS-PSYC) PHYS T1 vs. T2 (PHYS_ T1-PHYS_T2) Grad vs. Undergrad (PSYC_grad -PSYC_ undergrad) Mean difference Effect size D Mean difference Effect size D Mean difference Effect size D 1. ChatGPT is easy to use -0.38* -0.43 -0.44* -0.54 0.36* 0.41 2. ChatGPT seems to be useful for my learning -0.69* -0.74 -0.32 -0.32 0.47* 0.60 3. Use of ChatGPT could improve my learning -0.56* -0.62 -0.39 -0.41 0.34 0.41 4. I have fun interacting with ChatGPT -0.44* -0.46 -0.34 -0.40 0.61* 0.63 5. The use of ChatGPT helps me connect ideas in new ways. -0.49* -0.53 -0.22 -0.25 0.64 6. use of ChatGPT help me develop confidence in the subject area -0.43* -0.43 -0.41 -0.43 0.28 0.28 7. The use of ChatGPT motivated me to learn certain new skills -0.53* -0.55 -0.29 -0.33 0.64* 0.65 8. Use of ChatGPT can be important supplement to the classroom -0.45* -0.44 -0.48* -0.49 0.49* 0.49 9. I think the use of ChatGPT in classroom is a good idea. -0.57* -0.57 -0.31 -0.28 0.41* 0.46 10. I think ChatGPT provides a fair education opportunity to all the students -0.39* -0.34 -0.58* -0.51 0.13 0.12 11. I plan to or will continue to use ChatGPT in the future -0.75* -0.66 -0.57* -0.49 0.55* 0.53 12. I am satisfied with the ChatGPT activity -0.34* -0.35 -0.53* -0.58 0.33 0.33 13. I think ChatGPT can better address my individual questions than teachers -0.05 -0.05 0.32 0.27 -0.21 -0.20 14. I have some concerns over the use of ChatGPT in classrooms 0.26 0.22 0.41 0.36 0.16 0.14 15. Using ChatGPT takes too much of my time 0.33* 0.40 0.32 0.39 -0.31 -0.40 16. I am not comfortable with the idea of use of ChatGPT 0.37* 0.30 0.62* 0.50 -0.52* -0.46
Total n = 169. *For significant p-values less than.05. T1 refers to the Spring semester of 2023, and T2 refers to the Fall semester of 2023. The comparisons between graduate and undergraduate students are among psychology classroom. Among all the items, no significant differences were found between male vs. female, Whites vs. others, and first generation vs non-first generation college students.
the type of activity significantly influences student reactions to ChatGPT as an instructional aid tool. It is found that the coding activity in psychology classes was better received by students compared to the conceptual discussion activity in physics. Almost all survey items indicated significant differences, except for the items "better addresses my individual questions than teachers" and "have some concerns over the use of ChatGPT in classrooms." Among all the items, perceptions of usefulness and intention for future use showed the largest differences, with Cohen's d effect sizes of 0.74 and 0.66, respectively. These findings suggest that the type of activity plays a crucial role in shaping students' perceptions.
Coding activities may have been perceived as more valuable because they offer more tangible and structured learning opportunities compared to the nature of conceptual discussions.
For the psychology coding activity, the perceptions of participating students were similar across two semesters, with no significant differences in responses except for one item: "I think ChatGPT provides a fair education opportunity to all the students." For this item, student ratings increased from the first to the second semester by a rating difference of 0.60 out of a five-point Likert scale of agreement (Cohen's d of 0.56). This change may indicate that as students became more familiar with ChatGPT over time, they developed a greater appreciation for its ability to provide equitable learning opportunities and effectively address individual questions. Additionally, although no other items differed significantly over time, there was a general trend toward improved student perceptions of ChatGPT. This improvement may reflect the important influence of sustained exposure to GAI tools in fostering positive attitudes and maximizing their potential in educational settings.
For the discussion activity in physics courses, students' perceptions improved significantly over time. Notably, significant changes with modest effect sizes were observed in several areas, including ease of use, usefulness as a supplemental tool to classroom instruction, provision of fair educational opportunities, activity satisfaction, intent for future use, and comfort level with using ChatGPT. These findings suggest the importance of selfexploration and the dynamic interactions within student group activities, which appear to play a crucial role in enhancing both engagement and learning outcomes when integrating ChatGPT into classroom activities.
Comparisons between graduate and undergraduate students in the participating psychology courses reveal significant differences in the perceived effectiveness and usefulness of the GAI. Graduate students showed greater motivation to use ChatGPT for learning new skills (Cohen's d = 0.65) and reported a higher level of intention to use the tool in the future (Cohen's d = 0.53) compared to undergraduates. Additionally, we examined the impact of various demographic factors on students' reactions to ChatGPT among all students as well as within each discipline. These analyses suggest that there are no significant differences between male and female students, white students and students of other races, or first-generation and nonfirst-generation college students across the items of our evaluation.
Before presenting a general discussion, it is invaluable to consider individual instructors' reflections based on their year-long observations and experiences with the activities described here. These reflections provide a unique and nuanced perspective that complements the students' feedback.
For the coding activity, clear instructions and a well-thoughtout design were crucial in fostering students' high levels of engagement and perceived learning effectiveness. Several key observations are worth noting. First, regarding prior interaction experiences, most graduate students had some level of familiarity with ChatGPT. However, we were surprised to learn that many undergraduate students, even in the fall of 2023, were still unfamiliar with or had no prior experience using ChatGPT (i.e., 13 out of 25 psychology undergraduates, or 52%, had never used ChatGPT in September 2023). Despite this lower-thanexpected familiarity with ChatGPT, its ease of use ensured that participants' lack of experience did not affect their engagement in the experimental activity. On the other side, while ChatGPT proved accessible, other tools used in the activity, such as Google Colab, required more technical support from the instructor. Providing a thorough introduction and a step-by-step walkthrough of related operational tasks (e.g., creating, modifying, or running code) was essential to ensure students could effectively use these tools.
Second, despite their overall comfort and satisfaction with the ChatGPT integrated activity, students expressed some reservations about its use in a classroom setting (e.g., concerns over privacy and inaccurate information). These concerns underscore the need for careful consideration of how GAI is integrated into learning environments. Finally, although the instructor anticipated that this new GAI tool could serve as a customized resource to meet students' individual needs and promote a fairer educational environment, it may take time to fully realize the potential. While GAI tools become more versatile and mature, students need to become more adept at effectively leveraging their capabilities.
To maximize engagement and effectiveness, it is crucial for instructors to design activities thoughtfully, ensuring they are engaging and provide opportunities for students to share their solutions and feedback. For instance, group activities are essential in this activity, as some students may struggle with new topics and could benefit from peer support. The support and feedback from the instructor are equally important. Additionally, the difficulty level of the tasks should be appropriately challenging, allowing students to effectively utilize the GAI tool to solve problems Guo et al. 10.3389/feduc.2025.1574477 while fostering deeper learning. There can also not be any priori expectation or assumption of uniform familiarity or comfort with GAI tools among diverse groups of students.
Using ChatGPT to teach the concept of phase, or physics concepts in general, offers several advantages. Its interactive nature allows students to engage dynamically with the material, asking questions and receiving tailored explanations that make complex ideas more accessible. The GAI tool's availability at any time supports continuous learning, enabling students to explore topics outside of class hours and revisit concepts as needed. Additionally, ChatGPT provides immediate feedback, which helps students correct misunderstandings quickly and tailor their learning experience to their specific needs.
One of the most significant lessons learned and, we believe, a key driver of the improved outcomes observed in the second semester, was the critical role of increased student engagement. Strategies such as group discussions and opportunities for self-exploration were crucial in enhancing the effectiveness of the activity. These elements not only encouraged collaboration and deeper understanding but also fostered a more active and participatory learning environment. Group discussions and opportunities for self-exploration were thus important in the successful integration of GAI tools into classroom settings.
While ChatGPT offers valuable interactive learning, it has limitations that can impact the depth of understanding for complex concepts. For instance, when explaining the intricacies of phase shifts in wave interference, ChatGPT might provide a basic overview without delving into the mathematical derivations and experimental evidence that underpin these phenomena. This could result in students having a superficial grasp of the material rather than a comprehensive understanding. Moreover, ChatGPT may struggle to fully understand the context of a student's question. For example, if a student asks, "How does phase affect the sound quality in a concert hall?" ChatGPT might provide a general explanation of phase and sound waves without addressing the specific acoustical design considerations and real-world examples relevant to concert halls. This misalignment can lead to responses that are less relevant to the student's actual query.
The text-based nature of ChatGPT's interactions also may not adequately support more visual learners. Consider a student who learns best through visual aids asking ChatGPT to explain how phase differences lead to constructive and destructive interference. While GAI can describe these concepts textually, it cannot provide diagrams or interactive simulations that visually depict overlapping waves and their resultant interference patterns. This can limit the effectiveness of the explanation for visual learners.
Additionally, there is a risk of students misinterpreting the responses of GAI tools. For instance, consider a student asking ChatGPT, "What happens when two waves are out of phase by 180 degrees?" ChatGPT might explain that the waves will cancel each other out due to destructive interference. While this is correct, the GAI might not elaborate on the conditions necessary for perfect cancelation, such as the requirement for the waves to have the same amplitude and frequency. If the student misunderstands this, they might incorrectly assume that any two out-of-phase waves will always completely cancel each other out, regardless of their other properties. This misinterpretation could lead to confusion when the student encounters real-world scenarios or more complex problems where these additional factors play a crucial role.
In summary, while ChatGPT can be a valuable supplementary tool for teaching the concept of physics, it is best used in conjunction with other educational methods to address its limitations and enhance the overall learning experience.
In this manuscript, we describe the implementation details and findings from two in-class activity experiments conducted over two semesters in a university environment. These experiments were designed to evaluate undergraduate and graduate student (and instructor) reactions to and experiences with ChatGPT in classroom settings across two different academic disciplines. Overall, the integration of ChatGPT into classroom teaching was well-received by students, though outcomes such as learning effectiveness varied depending on the type and purpose of the activity. When the goal was to learn a new coding language in a statistics course, ChatGPT proved to be a highly effective tool, demonstrating its usefulness to students. In contrast, while students' perceptions of the physics-based conceptual discussion activity were generally positive, they were less favorable compared to those of the coding-focused learning activity. It is also important to note that perceptions of the discussion activity improved significantly over time, primarily due to the incorporation of collaborative group activities that enhanced engagement and learning outcomes.
Graduate students appeared more ready and motivated to embrace ChatGPT as a tool for acquiring new skills compared to their undergraduate counterparts. At the same time, no significant differences in perceptions or reactions were found across other demographic categories, such as gender, race, or first-generation college status. Some of the key findings among all comparative analyses are visualized in Figure 3. Notably, there was an improvement in satisfaction with the physics discussion activity over time, a higher level of motivation among graduate students compared to undergraduates in the psychology coding activity, and a greater perceived usefulness of ChatGPT in coding-based activities compared to discussion-based activities.
The findings from this preliminary study highlight key areas for improving the use of GAI tools like ChatGPT in the classroom. First, although students generally expressed positive satisfaction with the experimental class-based learning activities, their satisfaction levels were modest, with mean ratings falling below four on a five-point Likert scale. This indicates considerable room for improvement, particularly in fostering greater student engagement. Specifically, ChatGPT did not motivate students to learn new skills to the level we had anticipated in the physics course. This may be due to the nature of the discussion activity, where ChatGPT's performance did not exceed students' expectations as an effective facilitator of a discussion in that setting. Meanwhile, the overall improvement in students' perceptions in the physics course likely reflected the impact of pedagogical adjustments. Likewise, contrary to our initial prediction that ChatGPT could and would be perceived to function effectively as a personalized tutor, students did not perceive it as significantly better in facilitating fair educational opportunities. It is noteworthy that, on average, students did not believe ChatGPT could address individual questions more effectively than their instructors. These findings suggest that while ChatGPT may be a valuable educational tool, its effectiveness depends on the context in which it is used, the nature of the learning activity, and students' perceptions of its role in their education. For instance, its impact may be limited when students lack adequate support, or when they do not perceive it as meaningfully enhancing their engagement.
Interestingly, while the present analyses did not identify significant subgroup differences among participants (i.e., male vs. female, White vs. minority, first-generation students vs. non-firstgeneration students), it is important to note that first-generation and minority students perceived use of ChatGPT as a fairer learning opportunity when compared to their counterparts. Additionally, minority students reported a higher level of confidence and motivation to learn new skills. Although these differences were not statistically significant, they suggest that GAI can be a helpful tool to address the special needs of disadvantaged groups, offering benefits not typically provided by traditional instructional resources. However, these promising trends come with a cautionary note: the easy accessibility of GAI tools should not overshadow the technical challenges that some students face. It is essential to provide adequate support to all students, particularly those who are less familiar with such technologies.
Instructors should thoughtfully plan how to integrate ChatGPT and similar AI tools into their teaching practices to maximize their potential benefits while addressing potential challenges. Based on our findings, we offer the following suggestions for designing effective classroom activities using GAI tools like ChatGPT:
Address technical challenges: Ensure students have adequate support to effectively use GAI tools. This may include providing clear instructions, step-by-step walkthroughs, and troubleshooting guidance to build student confidence in navigating these technologies.
Tackle pedagogical challenges: Integrate GAI in ways that enhance, rather than replace, traditional teaching methods. Activities should leverage AI to complement human instruction by encouraging critical thinking, collaboration, and engagement.
Stay updated on technological advancements: With the rapid pace of GAI development, it is crucial for instructors to stay informed about emerging tools and features. For example, Google Colab now includes the embedded Gemini chatbot for direct interactions. Guo et al. 10.3389/feduc.2025.1574477 Consider ethical concerns: Carefully examine and address issues such as privacy and potential bias in AI-generated content. Instructors should promote discussions around the responsible use of AI tools among students.
Overcome practical limitations: Be mindful of accessibility and cost-related barriers that may prevent widespread adoption of AI tools. Additionally, investing in ongoing teacher training is essential to equip instructors with the skills needed to effectively implement AI technologies.
Evaluate long-term impact: While GAI tools like ChatGPT show promise in enhancing student learning, their long-term impact on educational outcomes remains unclear. Further research is necessary to better understand how these tools influence learning processes and how to optimize their integration into diverse classroom settings.
By considering these factors, instructors can design GAIenhanced activities that are engaging, equitable, and effective, ensuring that generative AI tools like ChatGPT become valuable assets in modern education.
This study has several limitations. First, while it is exploratory in nature and focuses on how students engage with GAI in a classroom setting, particularly during a period when such tools were still relatively new and unfamiliar, it was not designed to rigorously evaluate the effectiveness of GAI on learning outcomes (e.g., including a control or comparison group, where the traditional teaching approach was used for the same learning objectives). In addition, certain potential confounding variables such as cohort differences were not fully considered into the study design. Therefore, while the findings are informative, they should be interpreted with caution. Second, the study would benefit from being conducted on a larger scale, ideally including students from more diverse backgrounds to enhance the generalizability of the findings. At the same time, it is increasingly important to teach students to use GAI tools critically, effectively, and responsibly. Third, the reliance on self-report measures may introduce bias in evaluating learning outcomes. Future studies could incorporate more objective measures to provide a more robust assessment of students' learning and engagement. At the same time, the measures could be further refined, and the psychometric property could be validated to more robustly support the study's conclusions. Furthermore, the literature review could be expanded to include more databases, with a broader range of GAI tools beyond ChatGPT.
Frontiers in Education
frontiersin.org
In the spring semester of
2023, no group-based exploration tasks were included. Instead, the activities were instructor-led and demonstrated to the students. The activity outcome differences are discussed later in the manuscript.