skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


  • NSF Public Access
  • Search Results
  • Investigating effectiveness of distraction rate: augmented reality-based eye-tracking feature to predict student formative and summative performance
Title: Investigating effectiveness of distraction rate: augmented reality-based eye-tracking feature to predict student formative and summative performance
Augmented reality (AR) is gaining attraction as a valuable aid in training and educational settings. However, the cognitive overload due to the new learning environment may hamper effective learning during the AR sessions. For this reason, monitoring students learning status with an effective metric is required. Distraction rate (DR) is a feature extracted from a student’s eye-tracking coordinates data developed to measure the distracted proportion of a student in an AR learning session (Deay, 2023). In this study, we investigate DR with students’ formative and summative assessment outcomes to validate its effectiveness as a predictor for student performance.Methods: To do this, students learned a topic of biomechanics through several AR modules. The results of quizzes taken after each AR module and their class exam outcomes taken at the end of semesters provide formative and summative evaluation performances, respectively. The data were collected in two years in the same setup. To compute DR, the standard eye-tracking coordinates, called baseline, and those of an observed student are compared. In order to reduce false alarms, two sources of noise are accounted for. First, temporal noise caused by quick deviations from the baseline that only lasts for a short period of time is removed by computing the moving average of eye-tracking curves. Second, spatial noise caused by slight deviations in a student’s sight from the virtual instructor is reduced by applying a threshold to determine whether the deviation is large enough. Finally, the proportion of moving average signals exceeding the threshold is computed. Using mixed effects logistic regression models, this study shows how DR and students' performance are related while considering the year and student variations. To extract DR from eye-tracking data, two parameters should be determined, the window size and threshold. In this study, we carried out a comparison study with several parameter values with respect to the model’s prediction performance to find the best parameter tuning.Key Findings:For the formative performance, the results indicate that DR is a significant predictor for the probability of correct answers. For the summative performance, DR does not show a significant relationship with students’ exam scores, yet the negative regression coefficient of DR can be still found, indicating that the high DR value results in low performance in the exam. It can be interpreted that, due to the time interval between AR learning and exams, even if some students may have not paid much attention during the AR learning sessions, they could catch up on the material later by themselves. Overall, it is found that the exam performance is less sensitive, compared to the quiz performance, to students’ attention paid to AR learning sessions. Accordingly, the relationship between DR and summative performance is likely to be weaker than the case of formative assessment.  more » « less
Award ID(s):
2202108
PAR ID:
10643926
Author(s) / Creator(s):
; ; ; ; ;
Publisher / Repository:
AHFE
Date Published:
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. (, 52nd ACM Technical Symposium on Computer Science Education (SIGCSE ’21))
    null (Ed.)
    Unlike summative assessment that is aimed at grading students at the end of a unit or academic term, formative assessment is assess- ment for learning, aimed at monitoring ongoing student learning to provide feedback to both student and teacher, so that learning gaps can be addressed during the learning process. Education research points to formative assessment as a crucial vehicle for improving student learning. Formative assessment in K-12 CS and program- ming classrooms remains a crucial unaddressed need. Given that assessment for learning is closely tied to teacher pedagogical con- tent knowledge, formative assessment literacy needs to also be a topic of CS teacher PD. This position paper addresses the broad need to understand formative assessment and build a framework to understand the what, why, and how of formative assessment of introductory programming in K-12 CS. It shares specific pro- gramming examples to articulate the cycle of formative assessment, diagnostic evaluation, feedback, and action. The design of formative assessment items is informed by CS research on assessment design, albeit related largely to summative assessment and in CS1 contexts, and learning of programming, especially student misconceptions. It describes what teacher formative assessment literacy PD should entail and how to catalyze assessment-focused collaboration among K-12 CS teachers through assessment platforms and repositories. 
    more » « less
  2. ; ; ; (, Psychology of Mathematics Education North American)
    Olanoff, D; Johnson, K.; Spitzer, S (Ed.)
    Administrators, educators, and stakeholders have faced the dilemma of determining the most effective type of data for informing instruction for quite some time (Pella, 2015). While the type of standardized assessment a teacher gives during instruction is often set at the district or state level, teachers often have autonomy in the formative and summative assessments that serve as the day-to-day tools in assessing a student’s progress (Abrams et al., 2016). Choices about in-class assessment and instruction are building blocks towards a student’s success on standardized assessments. The purpose of this phenomenological qualitative study is to explore how 4th-8th grade math teachers’ preparation and instructional practices are influenced by the types of assessments administered to their students in one school. Research questions are as follows: (a) How do 4th-8th grade math teachers describe the math assessments they use? (b) How do 4th-8th grade math teachers adjust their instructional practices as a result of their students completing formative, summative, and standardized math assessments? 
    more » « less
  3. ; ; ; ; ; ; ; (, Proceedings of the Human Factors and Ergonomics Society Annual Meeting)
    This research aims to explore the prediction of student learning outcomes in Augmented Reality (AR) educational settings, focusing on engineering education, by analyzing pupil dilation and problem-solving time as key indicators. In this research, we have created an innovative AR learning platform through the incorporation of eye-tracking technology into the Microsoft HoloLens 2. This enhanced learning platform enables the collection of data on pupil dilation and problem-solving duration as students engage in AR-based learning activities. In this study, we hypothesize that pupil dilation and problem-solving time could be significant predictors of student performance in the AR learning environment. The results of our study suggest that problem-solving time may be a critical factor in predicting student learning success for materials involving procedural knowledge at low difficulty levels. Additionally, both pupil dilation and problem-solving time are predictive indicators of student learning outcomes when dealing with predominantly procedural knowledge at high difficulty levels. 
    more » « less
  4. (, Proceedings of the Linguistic Society of America)
    This paper describes a set of flipped learning materials that I created for emergency remote teaching of introductory linguistics. My goals were to create a set of asynchronous materials that would scaffold student progress through a syntax unit, require active engagement in the material, and enable students to receive incremental formative feedback. Assessment of formative and summative student progress in the unit demonstrates that these materials were as effective at supporting student learning as face-to-face pedagogical methods. The discussion touches on additional issues related to pedagogy of care that were overlooked. 
    more » « less
  5. ; ; ; ; (, Educational technology research and development)
    This investigation considers the relationship between test scores and a sorting task conceptual macrostructure measure based on topic-level term-term distances as Pathfinder networks. In Study 1 (n = 255), grade 7 Chinese students completed a sorting task 1 month after the traditional in-class lessons and exam. In Study 2 (n = 220), grade 8 students completed the sorting task immediately after self-directed study of a history text. In addition, a month later 68 of the students in Study 2 were further instructed to write a short essay about this content. Study 1 results showed significant correlations between the sorting task macrostructure network measures and both lesson and unit test scores. Study 2 obtained the same significant correlations between sorting task macrostructure network measures and performance on tests. In addition, in Study 2, essay conceptual networks of historical content were better for the high prior knowledge students. Both the sorting task and the essay writing task measures can complement traditional exam measures so that conceptual knowledge structure aspects of students’ learning can be identified for formative and summative purposes. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email