skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Student Perceptions of Fairness and Security in a Versioned Programming Exam
Using multiple versions of exams is a common exam security technique to prevent cheating in a variety of contexts. While psycho-metric techniques are routinely used by large high-stakes testing companies to ensure equivalence between exam versions, such approaches are generally cost and effort prohibitive for individual classrooms. As such, exam versions practically present a tension between exam security (which is enhanced by the versioning) and fairness (which results from difficulty variation between versions). In this work, we surveyed students on their perceptions of this trade-off between exam security and fairness on a versioned programming exam and found that significant populations value each aspect over the other. Furthermore, we found that students' expression of concerns about unfairness was not correlated to whether they had received harder versions of the course's most recent exam, but was correlated to lower overall course performance.  more » « less
Award ID(s):
1915257
PAR ID:
10200295
Author(s) / Creator(s):
;
Date Published:
Journal Name:
Proceedings of the 2020 ACM Conference on International Computing Education Research
Page Range / eLocation ID:
25 to 35
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; (, Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1)
    Battestilli, Lina; Rebelsky, Samuel A; Shoop, Libby (Ed.)
    We compare the exam security of three proctoring regimens of Bring-Your-Own-Device, synchronous, computer-based exams in a computer science class: online un-proctored, online proctored via Zoom, and in-person proctored. We performed two randomized crossover experiments to compare these proctoring regimens. The first study measured the score advantage students receive while taking un-proctored online exams over Zoom-proctored online exams. The second study measured the score advantage of students taking Zoom-proctored online exams over in-person proctored exams. In both studies, students took six 50-minute exams using their own devices, which included two coding questions and 8–10 non-coding questions. We find that students score 2.3% higher on non-coding questions when taking exams in the un-proctored format compared to Zoom proctoring. No statistically significant advantage was found for the coding questions. While most of the non-coding questions had randomization such that students got different versions, for the few questions where all students received the same exact version, the score advantage escalated to 5.2%. From the second study, we find no statistically significant difference between students’ performance on Zoom-proctored vs. in-person proctored exams. With this, we recommend educators incorporate some form of proctoring along with question randomization to mitigate cheating concerns in BYOD exams. 
    more » « less
  2. ; ; ; (, PLOS ONE)
    Carvalho, Paulo F. (Ed.)
    Evidence-based teaching practices are associated with improved student academic performance. However, these practices encompass a wide range of activities and determining which type, intensity or duration of activity is effective at improving student exam performance has been elusive. To address this shortcoming, we used a previously validated classroom observation tool, Practical Observation Rubric to Assess Active Learning (PORTAAL) to measure the presence, intensity, and duration of evidence-based teaching practices in a retrospective study of upper and lower division biology courses. We determined the cognitive challenge of exams by categorizing all exam questions obtained from the courses using Bloom’s Taxonomy of Cognitive Domains. We used structural equation modeling to correlate the PORTAAL practices with exam performance while controlling for cognitive challenge of exams, students’ GPA at start of the term, and students’ demographic factors. Small group activities, randomly calling on students or groups to answer questions, explaining alternative answers, and total time students were thinking, working with others or answering questions had positive correlations with exam performance. On exams at higher Bloom’s levels, students explaining the reasoning underlying their answers, students working alone, and receiving positive feedback from the instructor also correlated with increased exam performance. Our study is the first to demonstrate a correlation between the intensity or duration of evidence-based PORTAAL practices and student exam performance while controlling for Bloom’s level of exams, as well as looking more specifically at which practices correlate with performance on exams at low and high Bloom’s levels. This level of detail will provide valuable insights for faculty as they prioritize changes to their teaching. As we found that multiple PORTAAL practices had a positive association with exam performance, it may be encouraging for instructors to realize that there are many ways to benefit students’ learning by incorporating these evidence-based teaching practices. 
    more » « less
  3. ; ; ; (, American Society for Engineering Education)
    The current study examines the validity of the RESP diagnostic exam and its predictive validity relative to standardized tests with a sample of students (N = 976) who matriculated into Rice University from 2012 to 2014. The RESP diagnostic exam was related to grades, and we found that the correlation between the RESP diagnostic exam and grades was greater for STEM grades than non-STEM grades. We found that the diagnostic exam accounted for an incremental 9% of variance in STEM grades above SAT performance, but only 1% of incremental variance above SAT in non-STEM grades. Moreover, we found evidence of range restriction for both SAT and RESP diagnostic exam performance for Rice University matriculants, further suggesting the utility of the diagnostic exam is at the lower end of the distribution. In summary, our results suggest that an additional diagnostic exam written by schools to specifically measure STEM preparation for their program can be a useful addition to procedures for selecting students for special experiences such as summer bridge programs. 
    more » « less
  4. ; ; ; (, Proceedings of the Seventh ACM Conference on Learning @ Scale)
    null (Ed.)
    In this paper, we study a computerized exam system that allows students to attempt the same question multiple times. This system permits students either to receive feedback on their submitted answer immediately or to defer the feedback and grade questions in bulk. An analysis of student behavior in three courses across two semesters found similar student behaviors across courses and student groups. We found that only a small minority of students used the deferred feedback option. A clustering analysis that considered both when students chose to receive feedback and either to immediately retry incorrect problems or to attempt other unfinished problems identified four main student strategies. These strategies were correlated to statistically significant differences in exam scores, but it was not clear if some strategies improved outcomes or if stronger students tended to prefer certain strategies. 
    more » « less
  5. ; ; ; ; ; (, ACM)
    Introduction: The emergence and widespread adoption of generative AI (GenAI) chatbots such as ChatGPT, and programming assistants such as GitHub Copilot, have radically redefined the landscape of programming education. This calls for replication of studies and reexamination of findings from pre-GenAI CS contexts to understand the impact on students. Objectives: Achievement Goals are well studied in computing education and can be predictive of student interest and exam performance. The objective in this study is to compare findings from prior achievement goal studies in CS1 courses with new CS1 courses that emphasize the use of human-GenAI collaborative coding. Methods: In a CS1 course that integrates GenAI, we use linear regression to explore the relationship between achievement goals and prior experience on student interest, exam performance, and perceptions of GenAI. Results: As with prior findings in traditional CS1 classes, Mastery goals are correlated with interest in computing. Contradicting prior CS1 findings, normative goals are correlated with exam scores. Normative and mastery goals correlate with students’ perceptions of learning with GenAI. Mastery goals weakly correlate with reading and testing code output from GenAI. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email