taln-ls2n/pararev

ParaRev: Building a dataset for Scientific Paragraph Revision annotated with revision instruction

About

This repository contains ParaRev, a dataset of 48k revised scientific paragraphs with an evaluation subset of 641 paragraphs manually annotated with revision instructions. This dataset is extracted from the **CASIMIR** corpus, the extraction, and annotation process is described in:

ParaRev : Building a dataset for Scientific Paragraph Revision annotated with revision instruction (Jourdan et al., WRAICOGS 2025)

Content

The dataset is composed of two subsets:

• —pararev_full: The full dataset, composed of 48k pairs of revised paragraphs without annotation.
• —pararev_annot_subset: The manually annotated subset composed of 641 paragraphs, each paragraph have 2 annotations. Those paragraphs are also included in pararev_full

The data in pararev_full follow this distribution: | Distribution | # chars Src| # chars Tgt | # words Src | # words Tgt| # sents Src | # sents Tgt | % words deleted | % words added | Lev dist| | :--------- | ----------:| -----------:| --------: | ----------:| ------: | -------: | -------: | -------: | -------:| | Min | 47 | 48 | 7 | 7 | 1 | 1 | 0 | 0 | 0 | | Avg | 680.16 | 715.58 | 125.54 | 132.99 | 5.26 | 5.50 | 21.54 | 25.63 | 194.80 | | Max | 5202 | 5588 | 1003 | 1147 | 70 | 68 | 96.51 | 97.90 | 2265 | | Avg | 374.11 | 394.20 | 69.04 | 73.32 | 3.07 | 3.19 | 18.19 | 18.15 | 160.10 |

The data in pararev_annot_subset are labelled with the following distribution: | Label | Rewriting light| Rewriting medium | Rewriting heavy | Development | Content add | Content subs | Concision | Content del | Unusable| | :-----| ----------: | -----------: | --------: | ----------: | ------: | -------: | -------: | -------: | -------:| | Prct %| 15.44 | 14.27 | 4.13 | 19.07 | 12.99 | 6.47 | 12.83 | 4.72 | 10.06 |

The following data fields are available:

• —WIP

Please cite this work as:

@inproceedings{jourdan-etal-2025-pararev,
    title = "{P}ara{R}ev : Building a dataset for Scientific Paragraph Revision annotated with revision instruction",
    author = "Jourdan, L{\'e}ane  and
      Boudin, Florian  and
      Dufour, Richard  and
      Hernandez, Nicolas  and
      Aizawa, Akiko",
    editor = "Zock, Michael  and
      Inui, Kentaro  and
      Yuan, Zheng",
    booktitle = "Proceedings of the First Workshop on Writing Aids at the Crossroads of AI, Cognitive Science and NLP (WRAICOGS 2025)",
    month = jan,
    year = "2025",
    address = "Abu Dhabi, UAE",
    publisher = "International Committee on Computational Linguistics",
    url = "https://aclanthology.org/2025.wraicogs-1.4/",
    pages = "35--44",
    abstract = "Revision is a crucial step in scientific writing, where authors refine their work to improve clarity, structure, and academic quality. Existing approaches to automated writing assistance often focus on sentence-level revisions, which fail to capture the broader context needed for effective modification. In this paper, we explore the impact of shifting from sentence-level to paragraph-level scope for the task of scientific text revision. The paragraph level definition of the task allows for more meaningful changes, and is guided by detailed revision instructions rather than general ones. To support this task, we introduce ParaRev, the first dataset of revised scientific paragraphs with an evaluation subset manually annotated with revision instructions. Our experiments demonstrate that using detailed instructions significantly improves the quality of automated revisions compared to general approaches, no matter the model or the metric considered."
}