skip to main content


Title: Effect of Immediate Feedback on Math Achievement at the High School Level
We examine the use of computer-based learning in the classroom and the effect of immediate feedback on student performance. Since it is well known in educational research that it is possible to observe a “Matthew Effect” in which the rich get richer, we wanted to see if feedback was useful for low prior knowledge students, as defined by students whose pretest score was at or below the median. In this counterbalanced randomized controlled trial, 243 tenth and eleventh grade mathematics students were exposed to one of two conditions, as we measured their learning from: 1) immediate feedback (where the computer told them correctness and they could also ask for hints) or 2) practice only (where they received feedback only after taking a posttest). Results suggest that immediate feedback from computer-based learning tasks benefit both high and low prior knowledge students, with low prior knowledge students exhibiting greater gains. The implications of these findings support further investigation into the use of computer-based learning tasks that provide immediate feedback.  more » « less
Award ID(s):
1931419 1724889
NSF-PAR ID:
10191711
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
Artificial Intelligence in Education. AIED 2020. Lecture Notes in Computer Science
Volume:
12164
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. We examine the use of computer-based learning in the classroom and the effect of immediate feedback on student performance. Since it is well known in educational research that it is possible to observe a “Matthew Effect” in which the rich get richer, we wanted to see if feedback was useful for low prior knowledge students, as defined by students whose pretest score was at or below the median. In this counterbalanced randomized controlled trial, 243 tenth and eleventh grade mathematics students were exposed to one of two conditions, as we measured their learning from: 1) immediate feedback (where the computer told them correctness and they could also ask for hints) or 2) practice only (where they received feedback only after taking a posttest). Results suggest that immediate feedback from computer-based learning tasks benefit both high and low prior knowledge students, with low prior knowledge students exhibiting greater gains. The implications of these findings support further investigation into the use of computer-based learning tasks that provide immediate feedback. 
    more » « less
  2. Chen, J.Y.C. ; Fragomeni, G. (Ed.)
    Virtual Access to STEM Careers (VASC) is a technology-rich, inquiry and problem-based curriculum designed to expose and stimulate student interest in marine, environmental, computer, and geological sciences. Intended for 3rd through 5th grade students, VASC builds academic momentum at the intermediate level to prepare students for STEM opportunities later in middle school and high school. Our program is aligned with “Next Generation Science Standards” and “Common Core State Standards” and immerses students in rigorous, high-interest learning modules where students are introduced to and take on the roles of different STEM occupations. We are specifically developing and testing virtual reality-based modules that place students in a coastal environment where they learn about the sea turtle life-cycle. Students also practice the types of measurements and conservation tasks that park rangers and marine scientists regularly perform. The investigations focused on the design of a user interface that meets the needs of students and their teachers. We collected feedback on user interface design and knowledge gained by the users from the simulation. Additionally, we compared two different virtual reality head-mounted displays; i) HTC Vive and ii) Oculus Quest 2, to identify the pros and cons of each technology in future classroom settings. Our investigations yielded valuable information about how instructions should be presented to users, how the interface should provide immediate feedback for user error, how surveys should be administered, what equipment is most efficient for transporting and setting up large scale experiments in schools, and what types of interactions students and teachers want to experience in VASC. 
    more » « less
  3. Students come to learning in engineering classrooms with misconceptions about the concepts covered in engineering course contents. However, instructional efforts often do not effectively address misconceptions in students’ prior knowledge. Concept inventories (CIs) are often relied upon to identify misconceptions in students’ prior knowledge. However, many instructors never benefit much from using CIs because they lack either the know-how, time commitment, or statistical skills required to use them efficiently and effectively. Furthermore, there sometimes are ambiguities about how to interpret students’ CI scores. The Electric Circuit Concepts Diagnostic (ECCD) project team will address these limitations of CIs by creating a web-based electric circuit concept inventory that: (i.) provides an immediate and multipurpose feedback system for reporting about students’ circuits and electricity prior knowledge; (ii.) differentiates, with a high probability, between a lack of prior knowledge and misconceptions; and (iii.) uses a scheme of multidimensional knowledge profiles to report on students’ prior knowledge and misconceptions. The project will integrate the affordances of cognitive diagnostic modeling (CDM), multitier testing frameworks, and computer-assisted testing to realize these project objectives. This work-in-progress report introduces the objectives of ECCD inventory to the ECE research and teaching community. 
    more » « less
  4. Step-based tutoring systems, in which each step of a student’s work is accepted by a computer using special interfaces and provided immediate feedback, are known to be more effective in promoting learning than traditional and more common answer-based tutoring systems, in which only the final (usually numerical) answer is evaluated. Prior work showed that this approach can be highly effective in the domain of linear circuit analysis in teaching topics involving relatively simple solution procedures. Here, we demonstrate a novel application of this approach to more cognitively complex, multi-step procedures used to analyze linear circuits using the superposition and source transformation methods. Both methods require that students interactively edit a circuit diagram repeatedly, interspersed with the writing of relevant equations. Scores on post-tests and student opinions are compared using a blind classroom-based experiment where students are randomly assigned to use either the new system or a commercially published answer-based tutoring system on these topics. Post-test scores are not statistically significantly different but students prefer the step-based system by a margin of 84 to 11% for superposition and 68 to 23% for source transformations. 
    more » « less
  5. Jovanovic, Jelena ; Chounta, Irene-Angelica ; Uhomoibhi, James ; McLaren, Bruce (Ed.)
    Computer-supported education studies can perform two important roles. They can allow researchers to gather important data about student learning processes, and they can help students learn more efficiently and effectively by providing automatic immediate feedback on what the students have done so far. The evaluation of student work required for both of these roles can be relatively easy in domains like math, where there are clear right answers. When text is involved, however, automated evaluations become more difficult. Natural Language Processing (NLP) can provide quick evaluations of student texts. However, traditional neural network approaches require a large amount of data to train models with enough accuracy to be useful in analyzing student responses. Typically, educational studies collect data but often only in small amounts and with a narrow focus on a particular topic. BERT-based neural network models have revolutionized NLP because they are pre-trained on very large corpora, developing a robust, contextualized understanding of the language. Then they can be “fine-tuned” on a much smaller set of data for a particular task. However, these models still need a certain base level of training data to be reasonably accurate, and that base level can exceed that provided by educational applications, which might contain only a few dozen examples. In other areas of artificial intelligence, such as computer vision, model performance on small data sets has been improved by “data augmentation” — adding scaled and rotated versions of the original images to the training set. This has been attempted on textual data; however, augmenting text is much more difficult than simply scaling or rotating images. The newly generated sentences may not be semantically similar to the original sentence, resulting in an improperly trained model. In this paper, we examine a self-augmentation method that is straightforward and shows great improvements in performance with different BERT-based models in two different languages and on two different tasks that have small data sets. We also identify the limitations of the self-augmentation procedure. 
    more » « less