Search for: All records

This study investigates the implementation of a classroom response system in STEM education in a higher education context. The study used ExplainIt, a web-based classroom response system designed to support students’ self-explanations and provide instant feedback. Data were collected from 32 undergraduate students using four instruments including demographic information, self-efficacy, engagement, and system evaluation. The results showed that students reported positive learning experiences, demonstrated increased self-efficacy in STEM content, and indicated high levels of engagement following their use of ExplainIt. 

The practice of soliciting self-explanations from students is widely recognized for its pedagogical benefits. However, the labor-intensive effort required to manually assess students’ explanations makes it impractical for classroom settings. As a result, many current solutions to gauge students’ understanding during class are often limited to multiple choice or fill-in-the-blank questions, which are less effective at exposing misconceptions or helping students to understand and integrate new concepts. Recent advances in large language models (LLMs) present an opportunity to assess student explanations in real-time, making explanation-based classroom response systems feasible for implementation. In this work, we investigate LLM-based approaches for assessing the correctness of students’ explanations in response to undergraduate computer science questions. We investigate alternative prompting approaches for multiple LLMs (i.e., Llama 2, GPT-3.5, and GPT-4) and compare their performance to FLAN-T5 models trained in a fine-tuning manner. The results suggest that the highest accuracy and weighted F1 score were achieved by fine-tuning FLAN-T5, while an in-context learning approach with GPT-4 attains the highest macro F1 score.