skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


Title: Essay Revision and Corresponding Grade Change as Captured by Text Similarity and Revision Purposes
Writing and revision are abstract skills that can be challenging to teach to students. Automatic essay revision assistants offer to help in this area because they compare two drafts of a student's essay and analyze the revisions performed. For these assistants to be useful, they need to provide useful information such as whether the revisions are likely to lead to an improvement in the student's grade. It is necessary to better understand the connection between revisions and grade change so that this information could be displayed in an assistant. So, this work explores the relationship between the tf-idf cosine similarity of two essay drafts and resulting essay grade change. Prior work has demonstrated that identifying the revisions between drafts, then labeling each revision with the purpose behind why the revision was performed is useful to predicting grade change. However, this process is expensive because this sort of annotation is time-consuming for humans. Moreover, classifiers achieve lower accuracy than humans when predicting purposes. Using similarity measures instead of or as supplement to revision purposes may correct these issues, as similarity can be computed automatically and without the issue of classification accuracy. As such, the correlations between grade change and the similarity measure are compared to the correlations between grade change and revision purposes with the potential use-case of an automatic writing assistant in mind. Findings suggest tf-idf cosine similarity captures overall essay and overall grade change while revision purposes capture lighter changes that fix errors or cause the essay to read better.  more » « less
Award ID(s):
1735752
PAR ID:
10379506
Author(s) / Creator(s):
;
Date Published:
Journal Name:
Workshop for Undergraduates in Educational Data Mining and Learning Engineering
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; (, CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems)
    null (Ed.)
    We present the design and evaluation of a web-based intelligent writing assistant that helps students recognize their revisions of argumentative essays. To understand how our revision assistant can best support students, we have implemented four versions of our system with differences in the unit span (sentence versus sub-sentence) of revision analysis and the level of feedback provided (none, binary, or detailed revision purpose categorization). We first discuss the design decisions behind relevant components of the system, then analyze the efficacy of the different versions through a Wizard of Oz study with university students. Our results show that while a simple interface with no revision feedback is easier to use, an interface that provides a detailed categorization of sentence-level revisions is the most helpful based on user survey data, as well as the most effective based on improvement in writing outcomes. 
    more » « less
  2. ; ; (, Reading and Writing)
    Revision plays an important role in writing, and as revisions break down the linearity of the writing process, they are crucial in describing writing process dynamics. Keystroke logging and analysis have been used to identify revisions made during writing. Previous approaches include the manual annotation of revisions, building nonlinear S-notations, and the automated extraction of backspace keypresses. However, these approaches are time-intensive, vulnerable to construct, or restricted. Therefore, this article presents a computational approach to the automatic extraction of full revision events from keystroke logs, including both insertions and deletions, as well as the characters typed to replace the deleted text. Within this approach, revision candidates are first automatically extracted, which allows for a simplified manual annotation of revision events. Second, machine learning is used to automatically detect revision events. For this, 7120 revision events were manually annotated in a dataset of keystrokes obtained from 65 students conducting a writing task. The results showed that revision events could be automatically predicted with a relatively high accuracy. In addition, a case study proved that this approach could be easily applied to a new dataset. To conclude, computational approaches can be beneficial in providing automated insights into revisions in writing. 
    more » « less
  3. ; ; ; ; (, Written Communication)
    Peer review is useful for providing students with formative feedback, yet it is used less frequently in STEM classrooms and for supporting writing-to-learn (WTL). While research indicates the benefits of incorporating peer review into classrooms, less research is focused on students’ perceptions thereof. Such research is important as it speaks to the mechanisms whereby peer review can support learning. This study examines students’ self-reported approaches to and perceptions of peer review and revision associated with WTL assignments implemented in an organic chemistry course. Students responded to a survey covering how they approached peer review and revision and the benefits they perceived from participating in each. Findings indicate that the assignment materials guided students’ approaches during both peer review and revision. Furthermore, students described various ways both receiving feedback from their peers and reading their peers’ drafts were beneficial, but primarily connected their revisions to receiving feedback. 
    more » « less
  4. ; ; ; ; ; ; (, Association for Computational Linguistics)
    Dziri, Nouha; Ren, Sean; Diao, Shizhe (Ed.)
    The ability to revise essays in response to feedback is important for students’ writing success. An automated writing evaluation (AWE) system that supports students in revising their essays is thus essential. We present eRevise+RF, an enhanced AWE system for assessing student essay revisions (e.g., changes made to an essay to improve its quality in response to essay feedback) and providing revision feedback. We deployed the system with 6 teachers and 406 students across 3 schools in Pennsylvania and Louisiana. The results confirmed its effectiveness in (1) assessing student essays in terms of evidence usage, (2) extracting evidence and reasoning revisions across essays, and (3) determining revision success in responding to feedback. The evaluation also suggested eRevise+RF is a helpful system for young students to improve their argumentative writing skills through revision and formative feedback. 
    more » « less
  5. ; ; ; ; (, Educational technology research and development)
    This investigation considers the relationship between test scores and a sorting task conceptual macrostructure measure based on topic-level term-term distances as Pathfinder networks. In Study 1 (n = 255), grade 7 Chinese students completed a sorting task 1 month after the traditional in-class lessons and exam. In Study 2 (n = 220), grade 8 students completed the sorting task immediately after self-directed study of a history text. In addition, a month later 68 of the students in Study 2 were further instructed to write a short essay about this content. Study 1 results showed significant correlations between the sorting task macrostructure network measures and both lesson and unit test scores. Study 2 obtained the same significant correlations between sorting task macrostructure network measures and performance on tests. In addition, in Study 2, essay conceptual networks of historical content were better for the high prior knowledge students. Both the sorting task and the essay writing task measures can complement traditional exam measures so that conceptual knowledge structure aspects of students’ learning can be identified for formative and summative purposes. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email