Title: AI and formative assessment: The train has left the station
Abstract In response to Li, Reigh, He, and Miller's commentary,Can we and should we use artificial intelligence for formative assessment in science, we argue that artificial intelligence (AI) is already being widely employed in formative assessment across various educational contexts. While agreeing with Li et al.'s call for further studies on equity issues related to AI, we emphasize the need for science educators to adapt to the AI revolution that has outpaced the research community. We challenge the somewhat restrictive view of formative assessment presented by Li et al., highlighting the significant contributions of AI in providing formative feedback to students, assisting teachers in assessment practices, and aiding in instructional decisions. We contend that AI‐generated scores should not be equated with the entirety of formative assessment practice; no single assessment tool can capture all aspects of student thinking and backgrounds. We address concerns raised by Li et al. regarding AI bias and emphasize the importance of empirical testing and evidence‐based arguments in referring to bias. We assert that AI‐based formative assessment does not necessarily lead to inequity and can, in fact, contribute to more equitable educational experiences. Furthermore, we discuss how AI can facilitate the diversification of representational modalities in assessment practices and highlight the potential benefits of AI in saving teachers’ time and providing them with valuable assessment information. We call for a shift in perspective, from viewing AI as a problem to be solved to recognizing its potential as a collaborative tool in education. We emphasize the need for future research to focus on the effective integration of AI in classrooms, teacher education, and the development of AI systems that can adapt to diverse teaching and learning contexts. We conclude by underlining the importance of addressing AI bias, understanding its implications, and developing guidelines for best practices in AI‐based formative assessment. more »« less
Anaroua, Fadjimata I; Li, Qing; Liu, Hong
(, GIFTSymp 12)
Sinatra, Anne; Goldberg, Benjamin
(Ed.)
Over the past decade, the educational landscape has experienced a surge of online learning and instruc-tional platforms (Liu et al., 2020). This remarkable surge can be attributed to a confluence of factors, including the rising demand for higher education opportunities, the shortage of available teaching staff, and the rapid advancements in information technology and artificial intelligence capabilities. Artificial Intelligence (AI) remained a niche area of research with limited practical applications in education for over half a century (Bhutoria, 2022; Chen et al., 2020; Roll & Wylie, 2016) from 1950 to 2010. Howev-er, in recent years, the advent of Big Data and advancements in computing power have propelled AI into the educational mainstream (Alam, 2021; Chen et al., 2020; Hwang et al., 2020). Today, the rise of machine learning, deep learning, automation, together with advances in big data analysis has sparked novel perspectives and explorations around the potential of enhancing personalized learning, a long-term educational vision of technology-enhanced course options to meet student needs (Grant & Basye, 2014). Fostering personalized learning necessitates the development of digital learning environments that dynamically adapt to individual learners' knowledge, prior experiences, and interests, while effectively and efficiently guiding them towards achieving desired learning outcomes (Spector, 2014, 2016). AI-powered technologies have made it possible to analyze data generated by learners and provide instruc-tion that matches their learning performance. Through learning analytics and data mining techniques, large datasets collected are analyzed and processed to uncover learners' unique learning characteristics, often referred to as learner profiling (Tzouveli et al., 2008). Subsequently, leveraging artificial intelli-gence algorithms, the learning content is tailored, and personalized learning paths are designed to align with each learner's identified needs and preferences, thereby facilitating personalized learning experienc-es.
Walkington, Candace
(, School Science and Mathematics)
Generative artificial intelligence has become prevalent in discussions of educational technology, particularly in the context of mathematics education. These AI models can engage in human‐like conversation and generate answers to complex questions in real‐time, with education reports accentuating their potential to make teachers' work more efficient and improve student learning. This paper provides a review of the current literature on generative AI in mathematics education, focusing on four areas: generative AI for mathematics problem‐solving, generative AI for mathematics tutoring and feedback, generative AI to adapt mathematical tasks, and generative AI to assist mathematics teachers in planning. The paper discusses ethical and logistical issues that arise with the application of generative AI in mathematics education, and closes with some observations, recommendations, and future directions.
Walkington, C
(, Proceedings of the forty-sixth annual meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education)
Generative Artificial Intelligence has become prevalent in discussions of educational technology. These AI models can engage in human-like conversation and generate answers to complex questions in real-time, with education reports accentuating their potential to make teachers’ work more efficient and improve student learning. In this paper, I provide a review of the current literature on generative AI in mathematics education, focusing on four areas: generative AI for mathematics problem-solving, generative AI for mathematics tutoring and feedback, generative AI to adapt mathematical tasks, and generative AI to assist mathematics teachers in planning. I then discuss ethical and logistical issues that arise with the application of generative AI in mathematics education, and close with some observations, recommendations, and future directions for the field.
Cohn, Clayton; Snyder, Caitlin; Fonteles, Joyce_Horn; T_S, Ashwin; Montenegro, Justin; Biswas, Gautam
(, British Journal of Educational Technology)
AbstractRecent advances in generative artificial intelligence (AI) and multimodal learning analytics (MMLA) have allowed for new and creative ways of leveraging AI to support K12 students' collaborative learning in STEM+C domains. To date, there is little evidence of AI methods supporting students' collaboration in complex, open‐ended environments. AI systems are known to underperform humans in (1) interpreting students' emotions in learning contexts, (2) grasping the nuances of social interactions and (3) understanding domain‐specific information that was not well‐represented in the training data. As such, combined human and AI (ie, hybrid) approaches are needed to overcome the current limitations of AI systems. In this paper, we take a first step towards investigating how a human‐AI collaboration between teachers and researchers using an AI‐generated multimodal timeline can guide and support teachers' feedback while addressing students' STEM+C difficulties as they work collaboratively to build computational models and solve problems. In doing so, we present a framework characterizing the human component of our human‐AI partnership as a collaboration between teachers and researchers. To evaluate our approach, we present our timeline to a high school teacher and discuss the key insights gleaned from our discussions. Our case study analysis reveals the effectiveness of an iterative approach to using human‐AI collaboration to address students' STEM+C challenges: the teacher can use the AI‐generated timeline to guide formative feedback for students, and the researchers can leverage the teacher's feedback to help improve the multimodal timeline. Additionally, we characterize our findings with respect to two events of interest to the teacher: (1) when the students cross adifficulty threshold,and (2) thepoint of intervention, that is, when the teacher (or system) should intervene to provide effective feedback. It is important to note that the teacher explained that there should be a lag between (1) and (2) to give students a chance to resolve their own difficulties. Typically, such a lag is not implemented in computer‐based learning environments that provide feedback. Practitioner notesWhat is already known about this topicCollaborative, open‐ended learning environments enhance students' STEM+C conceptual understanding and practice, but they introduce additional complexities when students learn concepts spanning multiple domains.Recent advances in generative AI and MMLA allow for integrating multiple datastreams to derive holistic views of students' states, which can support more informed feedback mechanisms to address students' difficulties in complex STEM+C environments.Hybrid human‐AI approaches can help address collaborating students' STEM+C difficulties by combining the domain knowledge, emotional intelligence and social awareness of human experts with the general knowledge and efficiency of AI.What this paper addsWe extend a previous human‐AI collaboration framework using a hybrid intelligence approach to characterize the human component of the partnership as a researcher‐teacher partnership and present our approach as a teacher‐researcher‐AI collaboration.We adapt an AI‐generated multimodal timeline to actualize our human‐AI collaboration by pairing the timeline with videos of students encountering difficulties, engaging in active discussions with a high school teacher while watching the videos to discern the timeline's utility in the classroom.From our discussions with the teacher, we define two types ofinflection pointsto address students' STEM+C difficulties—thedifficulty thresholdand theintervention point—and discuss how thefeedback latency intervalseparating them can inform educator interventions.We discuss two ways in which our teacher‐researcher‐AI collaboration can help teachers support students encountering STEM+C difficulties: (1) teachers using the multimodal timeline to guide feedback for students, and (2) researchers using teachers' input to iteratively refine the multimodal timeline.Implications for practice and/or policyOur case study suggests that timeline gaps (ie, disengaged behaviour identified by off‐screen students, pauses in discourse and lulls in environment actions) are particularly important for identifying inflection points and formulating formative feedback.Human‐AI collaboration exists on a dynamic spectrum and requires varying degrees of human control and AI automation depending on the context of the learning task and students' work in the environment.Our analysis of this human‐AI collaboration using a multimodal timeline can be extended in the future to support students and teachers in additional ways, for example, designing pedagogical agents that interact directly with students, developing intervention and reflection tools for teachers, helping teachers craft daily lesson plans and aiding teachers and administrators in designing curricula.
Belghith, Yasmine; Riedl, Mark; Moore, Roxanne; Alemdar, Meltem; Roberts, Jessica
(, Information and Learning Sciences)
PurposeChallenges in teaching the engineering design process (EDP) at the high-school level, such as promoting good documentation practices, are well-documented. While developments in educational artificial intelligence (AI) systems have the potential to assist in addressing these challenges, the open-ended nature of the EDP leads to challenges that often lack the specificity required for actionable AI development. In addition, conventional educational AI systems (e.g. intelligent tutoring systems) primarily target procedural domain tasks with well-defined outcomes and problem-solving strategies, while the EDP involves open-ended problems and multiple correct solutions, making AI intervention timing and appropriateness complex. Design/methodology/approachAuthors conducted a six-week-long Research through Co-Design (RtCD) process (i.e. a co-design process rooted in Research through Design) with two experienced high-school engineering teachers to co-construct actionable insight in the form of AI intervention points (AI-IPs) in engineering education where an AI system can effectively intervene to support them while highlighting their pedagogical practices. FindingsThis paper leveraged the design of task models to iteratively refine our prior understanding of teachers’ experiences with teaching the EDP into three AI-IPs related to documentation, ephemeral interactions between teachers and students and disruptive failures that can serve as a focus for intelligent educational system designs. Originality/valueThis paper discusses the implications of these AI-IPs for designing educational AI systems to support engineering education as well as the importance of leveraging RtCD methodologies to engage teachers in developing intelligent educational systems that align with their needs and afford them control over computational interventions in their classrooms.
Zhai, Xiaoming, and Nehm, Ross H. AI and formative assessment: The train has left the station. Journal of Research in Science Teaching 60.6 Web. doi:10.1002/tea.21885.
Zhai, Xiaoming, & Nehm, Ross H. AI and formative assessment: The train has left the station. Journal of Research in Science Teaching, 60 (6). https://doi.org/10.1002/tea.21885
Zhai, Xiaoming, and Nehm, Ross H.
"AI and formative assessment: The train has left the station". Journal of Research in Science Teaching 60 (6). Country unknown/Code not available: Wiley Blackwell (John Wiley & Sons). https://doi.org/10.1002/tea.21885.https://par.nsf.gov/biblio/10433029.
@article{osti_10433029,
place = {Country unknown/Code not available},
title = {AI and formative assessment: The train has left the station},
url = {https://par.nsf.gov/biblio/10433029},
DOI = {10.1002/tea.21885},
abstractNote = {Abstract In response to Li, Reigh, He, and Miller's commentary,Can we and should we use artificial intelligence for formative assessment in science, we argue that artificial intelligence (AI) is already being widely employed in formative assessment across various educational contexts. While agreeing with Li et al.'s call for further studies on equity issues related to AI, we emphasize the need for science educators to adapt to the AI revolution that has outpaced the research community. We challenge the somewhat restrictive view of formative assessment presented by Li et al., highlighting the significant contributions of AI in providing formative feedback to students, assisting teachers in assessment practices, and aiding in instructional decisions. We contend that AI‐generated scores should not be equated with the entirety of formative assessment practice; no single assessment tool can capture all aspects of student thinking and backgrounds. We address concerns raised by Li et al. regarding AI bias and emphasize the importance of empirical testing and evidence‐based arguments in referring to bias. We assert that AI‐based formative assessment does not necessarily lead to inequity and can, in fact, contribute to more equitable educational experiences. Furthermore, we discuss how AI can facilitate the diversification of representational modalities in assessment practices and highlight the potential benefits of AI in saving teachers’ time and providing them with valuable assessment information. We call for a shift in perspective, from viewing AI as a problem to be solved to recognizing its potential as a collaborative tool in education. We emphasize the need for future research to focus on the effective integration of AI in classrooms, teacher education, and the development of AI systems that can adapt to diverse teaching and learning contexts. We conclude by underlining the importance of addressing AI bias, understanding its implications, and developing guidelines for best practices in AI‐based formative assessment.},
journal = {Journal of Research in Science Teaching},
volume = {60},
number = {6},
publisher = {Wiley Blackwell (John Wiley & Sons)},
author = {Zhai, Xiaoming and Nehm, Ross H.},
}
Warning: Leaving National Science Foundation Website
You are now leaving the National Science Foundation website to go to a non-government website.
Website:
NSF takes no responsibility for and exercises no control over the views expressed or the accuracy of
the information contained on this site. Also be aware that NSF's privacy policy does not apply to this site.
An official website of the United States government
