MichaelR207/MultiSimV2

1---2language:3- en4- fr5- ru6- ja7- it8- da9- es10- de11- pt12- sl13- ur14- eu15license: mit16size_categories:17- 1M<n<10M18task_categories:19- summarization20- text2text-generation21- text-generation22pretty_name: MultiSim23tags:24- medical25- legal26- wikipedia27- encyclopedia28- science29- literature30- news31- websites32configs:33- config_name: ASSET34  data_files:35  - split: train36    path: ASSET/train-*37  - split: validation38    path: ASSET/validation-*39  - split: test40    path: ASSET/test-*41- config_name: AdminIt42  data_files:43  - split: train44    path: AdminIt/train-*45  - split: validation46    path: AdminIt/validation-*47  - split: test48    path: AdminIt/test-*49- config_name: CLEAR50  data_files:51  - split: train52    path: CLEAR/train-*53  - split: validation54    path: CLEAR/validation-*55  - split: test56    path: CLEAR/test-*57- config_name: EasyJapanese58  data_files:59  - split: train60    path: EasyJapanese/train-*61  - split: validation62    path: EasyJapanese/validation-*63  - split: test64    path: EasyJapanese/test-*65- config_name: EasyJapaneseExtended66  data_files:67  - split: train68    path: EasyJapaneseExtended/train-*69  - split: validation70    path: EasyJapaneseExtended/validation-*71  - split: test72    path: EasyJapaneseExtended/test-*73- config_name: GEOLinoTest74  data_files:75  - split: train76    path: GEOLinoTest/train-*77  - split: validation78    path: GEOLinoTest/validation-*79  - split: test80    path: GEOLinoTest/test-*81- config_name: PaCCSS-IT82  data_files:83  - split: train84    path: PaCCSS-IT/train-*85  - split: validation86    path: PaCCSS-IT/validation-*87  - split: test88    path: PaCCSS-IT/test-*89- config_name: PorSimples90  data_files:91  - split: train92    path: PorSimples/train-*93  - split: validation94    path: PorSimples/validation-*95  - split: test96    path: PorSimples/test-*97- config_name: RSSE98  data_files:99  - split: train100    path: RSSE/train-*101  - split: validation102    path: RSSE/validation-*103  - split: test104    path: RSSE/test-*105- config_name: RuAdaptEncy106  data_files:107  - split: train108    path: RuAdaptEncy/train-*109  - split: validation110    path: RuAdaptEncy/validation-*111  - split: test112    path: RuAdaptEncy/test-*113- config_name: RuAdaptFairytales114  data_files:115  - split: train116    path: RuAdaptFairytales/train-*117  - split: validation118    path: RuAdaptFairytales/validation-*119  - split: test120    path: RuAdaptFairytales/test-*121- config_name: RuWikiLarge122  data_files:123  - split: train124    path: RuWikiLarge/train-*125  - split: validation126    path: RuWikiLarge/validation-*127  - split: test128    path: RuWikiLarge/test-*129- config_name: SimpitikiWiki130  data_files:131  - split: train132    path: SimpitikiWiki/train-*133  - split: validation134    path: SimpitikiWiki/validation-*135  - split: test136    path: SimpitikiWiki/test-*137- config_name: TSSlovene138  data_files:139  - split: train140    path: TSSlovene/train-*141  - split: validation142    path: TSSlovene/validation-*143  - split: test144    path: TSSlovene/test-*145- config_name: Teacher146  data_files:147  - split: train148    path: Teacher/train-*149  - split: validation150    path: Teacher/validation-*151  - split: test152    path: Teacher/test-*153- config_name: Terence154  data_files:155  - split: train156    path: Terence/train-*157  - split: validation158    path: Terence/validation-*159  - split: test160    path: Terence/test-*161- config_name: TextComplexityDE162  data_files:163  - split: train164    path: TextComplexityDE/train-*165  - split: validation166    path: TextComplexityDE/validation-*167  - split: test168    path: TextComplexityDE/test-*169- config_name: WikiAutoEN170  data_files:171  - split: train172    path: WikiAutoEN/train-*173  - split: validation174    path: WikiAutoEN/validation-*175  - split: test176    path: WikiAutoEN/test-*177- config_name: WikiLargeFR178  data_files:179  - split: train180    path: WikiLargeFR/train-*181  - split: validation182    path: WikiLargeFR/validation-*183  - split: test184    path: WikiLargeFR/test-*185dataset_info:186- config_name: ASSET187  features:188  - name: original189    dtype: string190  - name: simple191    sequence: string192  splits:193  - name: train194    num_bytes: 4293614195    num_examples: 19000196  - name: validation197    num_bytes: 123502198    num_examples: 100199  - name: test200    num_bytes: 411019201    num_examples: 359202  download_size: 0203  dataset_size: 4828135204- config_name: AdminIt205  features:206  - name: original207    dtype: string208  - name: simple209    sequence: string210  splits:211  - name: train212    num_bytes: 287476213    num_examples: 588214  - name: validation215    num_bytes: 31917216    num_examples: 48217  - name: test218    num_bytes: 31142219    num_examples: 49220  download_size: 0221  dataset_size: 350535222- config_name: CLEAR223  features:224  - name: original225    dtype: string226  - name: simple227    sequence: string228  splits:229  - name: train230    num_bytes: 1334806231    num_examples: 4196232  - name: validation233    num_bytes: 95136234    num_examples: 294235  - name: test236    num_bytes: 31618237    num_examples: 100238  download_size: 0239  dataset_size: 1461560240- config_name: EasyJapanese241  features:242  - name: original243    dtype: string244  - name: simple245    sequence: string246  splits:247  - name: train248    num_bytes: 4924255249    num_examples: 48000250  - name: validation251    num_bytes: 102150252    num_examples: 1000253  - name: test254    num_bytes: 101364255    num_examples: 999256  download_size: 0257  dataset_size: 5127769258- config_name: EasyJapaneseExtended259  features:260  - name: original261    dtype: string262  - name: simple263    sequence: string264  splits:265  - name: train266    num_bytes: 4458335267    num_examples: 33269268  - name: validation269    num_bytes: 139441270    num_examples: 1000271  - name: test272    num_bytes: 51193273    num_examples: 100274  download_size: 0275  dataset_size: 4648969276- config_name: GEOLinoTest277  features:278  - name: original279    dtype: string280  - name: simple281    sequence: string282  splits:283  - name: train284    num_bytes: 183396285    num_examples: 958286  - name: validation287    num_bytes: 18186288    num_examples: 82289  - name: test290    num_bytes: 20264291    num_examples: 81292  download_size: 0293  dataset_size: 221846294- config_name: PaCCSS-IT295  features:296  - name: original297    dtype: string298  - name: simple299    sequence: string300  splits:301  - name: train302    num_bytes: 5848946303    num_examples: 60485304  - name: validation305    num_bytes: 114014306    num_examples: 1061307  - name: test308    num_bytes: 114519309    num_examples: 1061310  download_size: 0311  dataset_size: 6077479312- config_name: PorSimples313  features:314  - name: original315    dtype: string316  - name: simple317    sequence: string318  splits:319  - name: train320    num_bytes: 1256127321    num_examples: 6290322  - name: validation323    num_bytes: 119369324    num_examples: 420325  - name: test326    num_bytes: 119016327    num_examples: 420328  download_size: 0329  dataset_size: 1494512330- config_name: RSSE331  features:332  - name: original333    dtype: string334  - name: simple335    sequence: string336  splits:337  - name: train338    num_bytes: 1347033339    num_examples: 3065340  - name: validation341    num_bytes: 81047342    num_examples: 85343  - name: test344    num_bytes: 850824345    num_examples: 1000346  download_size: 0347  dataset_size: 2278904348- config_name: RuAdaptEncy349  features:350  - name: original351    dtype: string352  - name: simple353    sequence: string354  splits:355  - name: train356    num_bytes: 2423702357    num_examples: 7782358  - name: validation359    num_bytes: 299291360    num_examples: 840361  - name: test362    num_bytes: 296144363    num_examples: 839364  download_size: 0365  dataset_size: 3019137366- config_name: RuAdaptFairytales367  features:368  - name: original369    dtype: string370  - name: simple371    sequence: string372  splits:373  - name: train374    num_bytes: 71036375    num_examples: 248376  - name: validation377    num_bytes: 7732378    num_examples: 31379  - name: test380    num_bytes: 7408381    num_examples: 31382  download_size: 0383  dataset_size: 86176384- config_name: RuWikiLarge385  features:386  - name: original387    dtype: string388  - name: simple389    sequence: string390  splits:391  - name: train392    num_bytes: 110393409393    num_examples: 246978394  - name: validation395    num_bytes: 325440396    num_examples: 678397  - name: test398    num_bytes: 151831399    num_examples: 312400  download_size: 0401  dataset_size: 110870680402- config_name: SimpitikiWiki403  features:404  - name: original405    dtype: string406  - name: simple407    sequence: string408  splits:409  - name: train410    num_bytes: 382373411    num_examples: 460412  - name: validation413    num_bytes: 42592414    num_examples: 52415  - name: test416    num_bytes: 47257417    num_examples: 51418  download_size: 0419  dataset_size: 472222420- config_name: TSSlovene421  features:422  - name: original423    dtype: string424  - name: simple425    sequence: string426  splits:427  - name: train428    num_bytes: 133723429    num_examples: 749430  - name: validation431    num_bytes: 16416432    num_examples: 94433  - name: test434    num_bytes: 15912435    num_examples: 96436  download_size: 28304437  dataset_size: 166051438- config_name: Teacher439  features:440  - name: original441    dtype: string442  - name: simple443    sequence: string444  splits:445  - name: train446    num_bytes: 29726447    num_examples: 136448  - name: validation449    num_bytes: 3822450    num_examples: 17451  - name: test452    num_bytes: 3983453    num_examples: 17454  download_size: 0455  dataset_size: 37531456- config_name: Terence457  features:458  - name: original459    dtype: string460  - name: simple461    sequence: string462  splits:463  - name: train464    num_bytes: 168652465    num_examples: 809466  - name: validation467    num_bytes: 20942468    num_examples: 102469  - name: test470    num_bytes: 19918471    num_examples: 101472  download_size: 0473  dataset_size: 209512474- config_name: TextComplexityDE475  features:476  - name: original477    dtype: string478  - name: simple479    sequence: string480  splits:481  - name: train482    num_bytes: 81994483    num_examples: 200484  - name: validation485    num_bytes: 11539486    num_examples: 25487  - name: test488    num_bytes: 8776489    num_examples: 25490  download_size: 0491  dataset_size: 102309492- config_name: WikiAutoEN493  features:494  - name: original495    dtype: string496  - name: simple497    sequence: string498  splits:499  - name: train500    num_bytes: 142873905501    num_examples: 576126502  - name: validation503    num_bytes: 1265282504    num_examples: 4988505  - name: test506    num_bytes: 1243704507    num_examples: 5002508  download_size: 0509  dataset_size: 145382891510- config_name: WikiLargeFR511  features:512  - name: original513    dtype: string514  - name: simple515    sequence: string516  splits:517  - name: train518    num_bytes: 80861778519    num_examples: 296402520  - name: validation521    num_bytes: 257078522    num_examples: 878523  - name: test524    num_bytes: 100283525    num_examples: 345526  download_size: 0527  dataset_size: 81219139528---529 530# Dataset Card for MultiSim Benchmark531 532## Dataset Description533 534- **Repository:https://github.com/XenonMolecule/MultiSim/tree/main** 535- **Paper:https://aclanthology.org/2023.acl-long.269/ https://arxiv.org/pdf/2305.15678.pdf** 536- **Point of Contact: michaeljryan@stanford.edu** 537 538### Dataset Summary539 540The MultiSim benchmark is a growing collection of text simplification datasets targeted at sentence simplification in several languages.  Currently, the benchmark spans 12 languages.541 542![Figure showing four complex and simple sentence pairs.  One pair in English, one in Japanese, one in Urdu, and one in Russian.  The English complex sentence reads "He settled in London, devoting himself chiefly to practical teaching." which is paired with the simple sentence "He lived in London. He was a teacher."](MultiSimEx.png "MultiSim Example")543 544### Supported Tasks545 546- Sentence Simplification547 548### Usage549 550```python551from datasets import load_dataset552 553dataset = load_dataset("MichaelR207/MultiSimV2")554```555 556### Citation557If you use this benchmark, please cite our [paper](https://aclanthology.org/2023.acl-long.269/):558```559@inproceedings{ryan-etal-2023-revisiting,560    title = "Revisiting non-{E}nglish Text Simplification: A Unified Multilingual Benchmark",561    author = "Ryan, Michael  and562      Naous, Tarek  and563      Xu, Wei",564    booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",565    month = jul,566    year = "2023",567    address = "Toronto, Canada",568    publisher = "Association for Computational Linguistics",569    url = "https://aclanthology.org/2023.acl-long.269",570    pages = "4898--4927",571    abstract = "Recent advancements in high-quality, large-scale English resources have pushed the frontier of English Automatic Text Simplification (ATS) research. However, less work has been done on multilingual text simplification due to the lack of a diverse evaluation benchmark that covers complex-simple sentence pairs in many languages. This paper introduces the MultiSim benchmark, a collection of 27 resources in 12 distinct languages containing over 1.7 million complex-simple sentence pairs. This benchmark will encourage research in developing more effective multilingual text simplification models and evaluation metrics. Our experiments using MultiSim with pre-trained multilingual language models reveal exciting performance improvements from multilingual training in non-English settings. We observe strong performance from Russian in zero-shot cross-lingual transfer to low-resource languages. We further show that few-shot prompting with BLOOM-176b achieves comparable quality to reference simplifications outperforming fine-tuned models in most languages. We validate these findings through human evaluation.",572}573```574 575### Contact576 577**Michael Ryan**: [Scholar](https://scholar.google.com/citations?user=8APGEEkAAAAJ&hl=en) | [Twitter](http://twitter.com/michaelryan207) | [Github](https://github.com/XenonMolecule) | [LinkedIn](https://www.linkedin.com/in/michael-ryan-207/) | [Research Gate](https://www.researchgate.net/profile/Michael-Ryan-86) | [Personal Website](http://michaelryan.tech/) | [michaeljryan@stanford.edu](mailto://michaeljryan@stanford.edu)578 579### Languages580 581- English582- French583- Russian584- Japanese585- Italian586- Danish (on request)587- Spanish (on request)588- German589- Brazilian Portuguese590- Slovene591- Urdu (on request)592- Basque (on request)593 594## Dataset Structure595 596### Data Instances597 598MultiSim is a collection of 27 existing datasets:599- AdminIT600- ASSET601- CBST602- CLEAR603- DSim604- Easy Japanese605- Easy Japanese Extended606- GEOLino607- German News608- Newsela EN/ES609- PaCCSS-IT610- PorSimples611- RSSE612- RuAdapt Encyclopedia613- RuAdapt Fairytales614- RuAdapt Literature615- RuWikiLarge616- SIMPITIKI617- Simple German618- Simplext619- SimplifyUR620- SloTS621- Teacher622- Terence623- TextComplexityDE624- WikiAuto625- WikiLargeFR626 627![Table 1: Important properties of text simplification parallel corpora](Table1.png "Table 1")628 629### Data Fields630 631In the train set, you will only find `original` and `simple` sentences.  In the validation and test sets you may find `simple1`, `simple2`, ... `simpleN` because a given sentence can have multiple reference simplifications (useful in SARI and BLEU calculations)632 633### Data Splits634 635The dataset is split into a train, validation, and test set.636 637![Table 2: MultiSim splits. *Original splits preserved](Table2.png "Table 2")638 639## Dataset Creation640 641### Curation Rationale642 643I hope that collecting all of these independently useful resources for text simplification together into one benchmark will encourage multilingual work on text simplification!644 645### Source Data646 647#### Initial Data Collection and Normalization648 649Data is compiled from the 27 existing datasets that comprise the MultiSim Benchmark.  For details on each of the resources please see Appendix A in the [paper](https://aclanthology.org/2023.acl-long.269.pdf).650 651#### Who are the source language producers?652 653Each dataset has different sources.  At a high level the sources are: Automatically Collected (ex. Wikipedia, Web data), Manually Collected (ex. annotators asked to simplify sentences), Target Audience Resources (ex. Newsela News Articles), or Translated (ex. Machine translations of existing datasets).654These sources can be seen in Table 1 pictured above (Section: `Dataset Structure/Data Instances`) and further discussed in section 3 of the [paper](https://aclanthology.org/2023.acl-long.269.pdf).  Appendix A of the paper has details on specific resources.655 656### Annotations657 658#### Annotation process659 660Annotators writing simplifications (only for some datasets) typically follow an annotation guideline.  Some example guidelines come from [here](https://dl.acm.org/doi/10.1145/1410140.1410191), [here](https://link.springer.com/article/10.1007/s11168-006-9011-1), and [here](https://link.springer.com/article/10.1007/s10579-017-9407-6).661 662#### Who are the annotators?663 664See Table 1 (Section: `Dataset Structure/Data Instances`) for specific annotators per dataset.  At a high level the annotators are: writers, translators, teachers, linguists, journalists, crowdworkers, experts, news agencies, medical students, students, writers, and researchers.665 666### Personal and Sensitive Information667 668No dataset should contain personal or sensitive information.  These were previously collected resources primarily collected from news sources, wikipedia, science communications, etc. and were not identified to have personally identifiable information.669 670## Considerations for Using the Data671 672### Social Impact of Dataset673 674We hope this dataset will make a greatly positive social impact as text simplification is a task that serves children, second language learners, and people with reading/cognitive disabilities.  By publicly releasing a dataset in 12 languages we hope to serve these global communities.675One negative and unintended use case for this data would be reversing the labels to make a "text complification" model.  We beleive the benefits of releasing this data outweigh the harms and hope that people use the dataset as intended.676 677### Discussion of Biases678 679There may be biases of the annotators involved in writing the simplifications towards how they believe a simpler sentence should be written.  Additionally annotators and editors have the choice of what information does not make the cut in the simpler sentence introducing information importance bias.680 681### Other Known Limitations682 683Some of the included resources were automatically collected or machine translated.  As such not every sentence is perfectly aligned.  Users are recommended to use such individual resources with caution.684 685## Additional Information686 687### Dataset Curators688 689**Michael Ryan**: [Scholar](https://scholar.google.com/citations?user=8APGEEkAAAAJ&hl=en) | [Twitter](http://twitter.com/michaelryan207) | [Github](https://github.com/XenonMolecule) | [LinkedIn](https://www.linkedin.com/in/michael-ryan-207/) | [Research Gate](https://www.researchgate.net/profile/Michael-Ryan-86) | [Personal Website](http://michaelryan.tech/) | [michaeljryan@stanford.edu](mailto://michaeljryan@stanford.edu)690 691### Licensing Information692 693MIT License694 695### Citation Information696 697Please cite the individual datasets that you use within the MultiSim benchmark as appropriate. Proper bibtex attributions for each of the datasets are included below.698 699#### AdminIT700```701@inproceedings{miliani-etal-2022-neural,702    title = "Neural Readability Pairwise Ranking for Sentences in {I}talian Administrative Language",703    author = "Miliani, Martina  and704      Auriemma, Serena  and705      Alva-Manchego, Fernando  and706      Lenci, Alessandro",707    booktitle = "Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing",708    month = nov,709    year = "2022",710    address = "Online only",711    publisher = "Association for Computational Linguistics",712    url = "https://aclanthology.org/2022.aacl-main.63",713    pages = "849--866",714    abstract = "Automatic Readability Assessment aims at assigning a complexity level to a given text, which could help improve the accessibility to information in specific domains, such as the administrative one. In this paper, we investigate the behavior of a Neural Pairwise Ranking Model (NPRM) for sentence-level readability assessment of Italian administrative texts. To deal with data scarcity, we experiment with cross-lingual, cross- and in-domain approaches, and test our models on Admin-It, a new parallel corpus in the Italian administrative language, containing sentences simplified using three different rewriting strategies. We show that NPRMs are effective in zero-shot scenarios ({\textasciitilde}0.78 ranking accuracy), especially with ranking pairs containing simplifications produced by overall rewriting at the sentence-level, and that the best results are obtained by adding in-domain data (achieving perfect performance for such sentence pairs). Finally, we investigate where NPRMs failed, showing that the characteristics of the training data, rather than its size, have a bigger effect on a model{'}s performance.",715}716```717 718#### ASSET719```720@inproceedings{alva-manchego-etal-2020-asset,721    title = "{ASSET}: {A} Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations",722    author = "Alva-Manchego, Fernando  and723      Martin, Louis  and724      Bordes, Antoine  and725      Scarton, Carolina  and726      Sagot, Beno{\^\i}t  and727      Specia, Lucia",728    booktitle = "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics",729    month = jul,730    year = "2020",731    address = "Online",732    publisher = "Association for Computational Linguistics",733    url = "https://www.aclweb.org/anthology/2020.acl-main.424",734    pages = "4668--4679",735}736```737#### CBST738```739@article{10.1007/s10579-017-9407-6,740  title={{The corpus of Basque simplified texts (CBST)}},741  author={Gonzalez-Dios, Itziar and Aranzabe, Mar{\'\i}a Jes{\'u}s and D{\'\i}az de Ilarraza, Arantza},742  journal={Language Resources and Evaluation},743  volume={52},744  number={1},745  pages={217--247},746  year={2018},747  publisher={Springer}748}749```750#### CLEAR751```752@inproceedings{grabar-cardon-2018-clear,753    title = "{CLEAR} {--} Simple Corpus for Medical {F}rench",754    author = "Grabar, Natalia  and755      Cardon, R{\'e}mi",756    booktitle = "Proceedings of the 1st Workshop on Automatic Text Adaptation ({ATA})",757    month = nov,758    year = "2018",759    address = "Tilburg, the Netherlands",760    publisher = "Association for Computational Linguistics",761    url = "https://aclanthology.org/W18-7002",762    doi = "10.18653/v1/W18-7002",763    pages = "3--9",764}765```766#### DSim767```768@inproceedings{klerke-sogaard-2012-dsim,769    title = "{DS}im, a {D}anish Parallel Corpus for Text Simplification",770    author = "Klerke, Sigrid  and771      S{\o}gaard, Anders",772    booktitle = "Proceedings of the Eighth International Conference on Language Resources and Evaluation ({LREC}'12)",773    month = may,774    year = "2012",775    address = "Istanbul, Turkey",776    publisher = "European Language Resources Association (ELRA)",777    url = "http://www.lrec-conf.org/proceedings/lrec2012/pdf/270_Paper.pdf",778    pages = "4015--4018",779    abstract = "We present DSim, a new sentence aligned Danish monolingual parallel corpus extracted from 3701 pairs of news telegrams and corresponding professionally simplified short news articles. The corpus is intended for building automatic text simplification for adult readers. We compare DSim to different examples of monolingual parallel corpora, and we argue that this corpus is a promising basis for future development of automatic data-driven text simplification systems in Danish. The corpus contains both the collection of paired articles and a sentence aligned bitext, and we show that sentence alignment using simple tf*idf weighted cosine similarity scoring is on line with state―of―the―art when evaluated against a hand-aligned sample. The alignment results are compared to state of the art for English sentence alignment. We finally compare the source and simplified sides of the corpus in terms of lexical and syntactic characteristics and readability, and find that the one―to―many sentence aligned corpus is representative of the sentence simplifications observed in the unaligned collection of article pairs.",780}781```782#### Easy Japanese783```784@inproceedings{maruyama-yamamoto-2018-simplified,785    title = "Simplified Corpus with Core Vocabulary",786    author = "Maruyama, Takumi  and787      Yamamoto, Kazuhide",788    booktitle = "Proceedings of the Eleventh International Conference on Language Resources and Evaluation ({LREC} 2018)",789    month = may,790    year = "2018",791    address = "Miyazaki, Japan",792    publisher = "European Language Resources Association (ELRA)",793    url = "https://aclanthology.org/L18-1185",794}795```796#### Easy Japanese Extended797```798@inproceedings{katsuta-yamamoto-2018-crowdsourced,799    title = "Crowdsourced Corpus of Sentence Simplification with Core Vocabulary",800    author = "Katsuta, Akihiro  and801      Yamamoto, Kazuhide",802    booktitle = "Proceedings of the Eleventh International Conference on Language Resources and Evaluation ({LREC} 2018)",803    month = may,804    year = "2018",805    address = "Miyazaki, Japan",806    publisher = "European Language Resources Association (ELRA)",807    url = "https://aclanthology.org/L18-1072",808}809```810#### GEOLino811```812@inproceedings{mallinson2020,813  title={Zero-Shot Crosslingual Sentence Simplification},814  author={Mallinson, Jonathan and Sennrich, Rico and Lapata, Mirella},815  year={2020},816  booktitle={2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020)}817}818```819#### German News820```821@inproceedings{sauberli-etal-2020-benchmarking,822    title = "Benchmarking Data-driven Automatic Text Simplification for {G}erman",823    author = {S{\"a}uberli, Andreas  and824      Ebling, Sarah  and825      Volk, Martin},826    booktitle = "Proceedings of the 1st Workshop on Tools and Resources to Empower People with REAding DIfficulties (READI)",827    month = may,828    year = "2020",829    address = "Marseille, France",830    publisher = "European Language Resources Association",831    url = "https://aclanthology.org/2020.readi-1.7",832    pages = "41--48",833    abstract = "Automatic text simplification is an active research area, and there are first systems for English, Spanish, Portuguese, and Italian. For German, no data-driven approach exists to this date, due to a lack of training data. In this paper, we present a parallel corpus of news items in German with corresponding simplifications on two complexity levels. The simplifications have been produced according to a well-documented set of guidelines. We then report on experiments in automatically simplifying the German news items using state-of-the-art neural machine translation techniques. We demonstrate that despite our small parallel corpus, our neural models were able to learn essential features of simplified language, such as lexical substitutions, deletion of less relevant words and phrases, and sentence shortening.",834    language = "English",835    ISBN = "979-10-95546-45-0",836}837```838#### Newsela EN/ES839```840@article{xu-etal-2015-problems,841    title = "Problems in Current Text Simplification Research: New Data Can Help",842    author = "Xu, Wei  and843      Callison-Burch, Chris  and844      Napoles, Courtney",845    journal = "Transactions of the Association for Computational Linguistics",846    volume = "3",847    year = "2015",848    address = "Cambridge, MA",849    publisher = "MIT Press",850    url = "https://aclanthology.org/Q15-1021",851    doi = "10.1162/tacl_a_00139",852    pages = "283--297",853    abstract = "Simple Wikipedia has dominated simplification research in the past 5 years. In this opinion paper, we argue that focusing on Wikipedia limits simplification research. We back up our arguments with corpus analysis and by highlighting statements that other researchers have made in the simplification literature. We introduce a new simplification dataset that is a significant improvement over Simple Wikipedia, and present a novel quantitative-comparative approach to study the quality of simplification data resources.",854}855```856#### PaCCSS-IT857```858@inproceedings{brunato-etal-2016-paccss,859    title = "{P}a{CCSS}-{IT}: A Parallel Corpus of Complex-Simple Sentences for Automatic Text Simplification",860    author = "Brunato, Dominique  and861      Cimino, Andrea  and862      Dell{'}Orletta, Felice  and863      Venturi, Giulia",864    booktitle = "Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing",865    month = nov,866    year = "2016",867    address = "Austin, Texas",868    publisher = "Association for Computational Linguistics",869    url = "https://aclanthology.org/D16-1034",870    doi = "10.18653/v1/D16-1034",871    pages = "351--361",872}873```874#### PorSimples875```876@inproceedings{aluisio-gasperin-2010-fostering,877    title = "Fostering Digital Inclusion and Accessibility: The {P}or{S}imples project for Simplification of {P}ortuguese Texts",878    author = "Alu{\'\i}sio, Sandra  and879      Gasperin, Caroline",880    booktitle = "Proceedings of the {NAACL} {HLT} 2010 Young Investigators Workshop on Computational Approaches to Languages of the {A}mericas",881    month = jun,882    year = "2010",883    address = "Los Angeles, California",884    publisher = "Association for Computational Linguistics",885    url = "https://aclanthology.org/W10-1607",886    pages = "46--53",887}888```889```890@inproceedings{10.1007/978-3-642-16952-6_31,891  author="Scarton, Carolina and Gasperin, Caroline and Aluisio, Sandra",892  editor="Kuri-Morales, Angel and Simari, Guillermo R.",893  title="Revisiting the Readability Assessment of Texts in Portuguese",894  booktitle="Advances in Artificial Intelligence -- IBERAMIA 2010",895  year="2010",896  publisher="Springer Berlin Heidelberg",897  address="Berlin, Heidelberg",898  pages="306--315",899  isbn="978-3-642-16952-6"900}901```902#### RSSE903```904@inproceedings{sakhovskiy2021rusimplesenteval,905  title={{RuSimpleSentEval-2021 shared task:} evaluating sentence simplification for Russian},906  author={Sakhovskiy, Andrey and Izhevskaya, Alexandra and Pestova, Alena and Tutubalina, Elena and Malykh, Valentin and Smurov, Ivana and Artemova, Ekaterina},907  booktitle={Proceedings of the International Conference “Dialogue},908  pages={607--617},909  year={2021}910}911```912#### RuAdapt913```914@inproceedings{Dmitrieva2021Quantitative,915  title={A quantitative study of simplification strategies in adapted texts for L2 learners of Russian},916  author={Dmitrieva, Anna and Laposhina, Antonina and Lebedeva, Maria},917  booktitle={Proceedings of the International Conference “Dialogue},918  pages={191--203},919  year={2021}920}921```922```923@inproceedings{dmitrieva-tiedemann-2021-creating,924    title = "Creating an Aligned {R}ussian Text Simplification Dataset from Language Learner Data",925    author = {Dmitrieva, Anna  and926      Tiedemann, J{\"o}rg},927    booktitle = "Proceedings of the 8th Workshop on Balto-Slavic Natural Language Processing",928    month = apr,929    year = "2021",930    address = "Kiyv, Ukraine",931    publisher = "Association for Computational Linguistics",932    url = "https://aclanthology.org/2021.bsnlp-1.8",933    pages = "73--79",934    abstract = "Parallel language corpora where regular texts are aligned with their simplified versions can be used in both natural language processing and theoretical linguistic studies. They are essential for the task of automatic text simplification, but can also provide valuable insights into the characteristics that make texts more accessible and reveal strategies that human experts use to simplify texts. Today, there exist a few parallel datasets for English and Simple English, but many other languages lack such data. In this paper we describe our work on creating an aligned Russian-Simple Russian dataset composed of Russian literature texts adapted for learners of Russian as a foreign language. This will be the first parallel dataset in this domain, and one of the first Simple Russian datasets in general.",935}936```937#### RuWikiLarge938```939@inproceedings{sakhovskiy2021rusimplesenteval,940  title={{RuSimpleSentEval-2021 shared task:} evaluating sentence simplification for Russian},941  author={Sakhovskiy, Andrey and Izhevskaya, Alexandra and Pestova, Alena and Tutubalina, Elena and Malykh, Valentin and Smurov, Ivana and Artemova, Ekaterina},942  booktitle={Proceedings of the International Conference “Dialogue},943  pages={607--617},944  year={2021}945}946```947#### SIMPITIKI948```949@article{tonelli2016simpitiki,950  title={SIMPITIKI: a Simplification corpus for Italian},951  author={Tonelli, Sara and Aprosio, Alessio Palmero and Saltori, Francesca},952  journal={Proceedings of CLiC-it},953  year={2016}954}955```956#### Simple German957```958@inproceedings{battisti-etal-2020-corpus,959    title = "A Corpus for Automatic Readability Assessment and Text Simplification of {G}erman",960    author = {Battisti, Alessia  and961      Pf{\"u}tze, Dominik  and962      S{\"a}uberli, Andreas  and963      Kostrzewa, Marek  and964      Ebling, Sarah},965    booktitle = "Proceedings of the Twelfth Language Resources and Evaluation Conference",966    month = may,967    year = "2020",968    address = "Marseille, France",969    publisher = "European Language Resources Association",970    url = "https://aclanthology.org/2020.lrec-1.404",971    pages = "3302--3311",972    abstract = "In this paper, we present a corpus for use in automatic readability assessment and automatic text simplification for German, the first of its kind for this language. The corpus is compiled from web sources and consists of parallel as well as monolingual-only (simplified German) data amounting to approximately 6,200 documents (nearly 211,000 sentences). As a unique feature, the corpus contains information on text structure (e.g., paragraphs, lines), typography (e.g., font type, font style), and images (content, position, and dimensions). While the importance of considering such information in machine learning tasks involving simplified language, such as readability assessment, has repeatedly been stressed in the literature, we provide empirical evidence for its benefit. We also demonstrate the added value of leveraging monolingual-only data for automatic text simplification via machine translation through applying back-translation, a data augmentation technique.",973    language = "English",974    ISBN = "979-10-95546-34-4",975}976```977#### Simplext978```979@article{10.1145/2738046,980    author = {Saggion, Horacio and \v{S}tajner, Sanja and Bott, Stefan and Mille, Simon and Rello, Luz and Drndarevic, Biljana},981    title = {Making It Simplext: Implementation and Evaluation of a Text Simplification System for Spanish},982    year = {2015},983    issue_date = {June 2015}, publisher = {Association for Computing Machinery},984    address = {New York, NY, USA},985    volume = {6},986    number = {4},987    issn = {1936-7228},988    url = {https://doi.org/10.1145/2738046},989    doi = {10.1145/2738046},990    journal = {ACM Trans. Access. Comput.},991    month = {may},992    articleno = {14},993    numpages = {36},994    keywords = {Spanish, text simplification corpus, human evaluation, readability measures} 995}996```997#### SimplifyUR998```999@inproceedings{qasmi-etal-2020-simplifyur,1000    title = "{S}implify{UR}: Unsupervised Lexical Text Simplification for {U}rdu",1001    author = "Qasmi, Namoos Hayat  and1002      Zia, Haris Bin  and1003      Athar, Awais  and1004      Raza, Agha Ali",1005    booktitle = "Proceedings of the Twelfth Language Resources and Evaluation Conference",1006    month = may,1007    year = "2020",1008    address = "Marseille, France",1009    publisher = "European Language Resources Association",1010    url = "https://aclanthology.org/2020.lrec-1.428",1011    pages = "3484--3489",1012    language = "English",1013    ISBN = "979-10-95546-34-4",1014}1015```1016#### SloTS1017```1018@misc{gorenc2022slovene,1019	 title = {Slovene text simplification dataset {SloTS}},1020	 author = {Gorenc, Sabina and Robnik-{\v S}ikonja, Marko},1021	 url = {http://hdl.handle.net/11356/1682},1022	 note = {Slovenian language resource repository {CLARIN}.{SI}},1023	 copyright = {Creative Commons - Attribution 4.0 International ({CC} {BY} 4.0)},1024	 issn = {2820-4042},1025	 year = {2022}1026}1027```1028#### Terence and Teacher1029```1030@inproceedings{brunato-etal-2015-design,1031    title = "Design and Annotation of the First {I}talian Corpus for Text Simplification",1032    author = "Brunato, Dominique  and1033      Dell{'}Orletta, Felice  and1034      Venturi, Giulia  and1035      Montemagni, Simonetta",1036    booktitle = "Proceedings of the 9th Linguistic Annotation Workshop",1037    month = jun,1038    year = "2015",1039    address = "Denver, Colorado, USA",1040    publisher = "Association for Computational Linguistics",1041    url = "https://aclanthology.org/W15-1604",1042    doi = "10.3115/v1/W15-1604",1043    pages = "31--41",1044}1045```1046#### TextComplexityDE1047```1048@article{naderi2019subjective,1049  title={Subjective Assessment of Text Complexity: A Dataset for German Language},1050  author={Naderi, Babak and Mohtaj, Salar and Ensikat, Kaspar and M{\"o}ller, Sebastian},1051  journal={arXiv preprint arXiv:1904.07733},1052  year={2019}1053}1054```1055#### WikiAuto1056```1057@inproceedings{acl/JiangMLZX20,1058  author    = {Chao Jiang and1059               Mounica Maddela and1060               Wuwei Lan and1061               Yang Zhong and1062               Wei Xu},1063  editor    = {Dan Jurafsky and1064               Joyce Chai and1065               Natalie Schluter and1066               Joel R. Tetreault},1067  title     = {Neural {CRF} Model for Sentence Alignment in Text Simplification},1068  booktitle = {Proceedings of the 58th Annual Meeting of the Association for Computational1069               Linguistics, {ACL} 2020, Online, July 5-10, 2020},1070  pages     = {7943--7960},1071  publisher = {Association for Computational Linguistics},1072  year      = {2020},1073  url       = {https://www.aclweb.org/anthology/2020.acl-main.709/}1074}1075```1076#### WikiLargeFR1077```1078@inproceedings{cardon-grabar-2020-french,1079    title = "{F}rench Biomedical Text Simplification: When Small and Precise Helps",1080    author = "Cardon, R{\'e}mi  and1081      Grabar, Natalia",1082    booktitle = "Proceedings of the 28th International Conference on Computational Linguistics",1083    month = dec,1084    year = "2020",1085    address = "Barcelona, Spain (Online)",1086    publisher = "International Committee on Computational Linguistics",1087    url = "https://aclanthology.org/2020.coling-main.62",1088    doi = "10.18653/v1/2020.coling-main.62",1089    pages = "710--716",1090    abstract = "We present experiments on biomedical text simplification in French. We use two kinds of corpora {--} parallel sentences extracted from existing health comparable corpora in French and WikiLarge corpus translated from English to French {--} and a lexicon that associates medical terms with paraphrases. Then, we train neural models on these parallel corpora using different ratios of general and specialized sentences. We evaluate the results with BLEU, SARI and Kandel scores. The results point out that little specialized data helps significantly the simplification.",1091}1092```1093 1094## Data Availability1095### Public Datasets1096Most of the public datasets are available as a part of this MultiSim Repo.  A few are still pending availability.  For all resources we provide alternative download links.1097| Dataset | Language | Availability in MultiSim Repo | Alternative Link |1098|---|---|---|---|1099| ASSET  | English | Available | https://huggingface.co/datasets/asset |1100| WikiAuto | English | Available | https://huggingface.co/datasets/wiki_auto |1101| CLEAR | French | Available | http://natalia.grabar.free.fr/resources.php#remi |1102| WikiLargeFR | French | Available | http://natalia.grabar.free.fr/resources.php#remi |1103| GEOLino | German | Available | https://github.com/Jmallins/ZEST-data |1104| TextComplexityDE | German | Available | https://github.com/babaknaderi/TextComplexityDE |1105| AdminIT | Italian | Available | https://github.com/Unipisa/admin-It |1106| Simpitiki | Italian | Available | https://github.com/dhfbk/simpitiki# |1107| PaCCSS-IT | Italian | Available | http://www.italianlp.it/resources/paccss-it-parallel-corpus-of-complex-simple-sentences-for-italian/ |1108| Terence and Teacher | Italian | Available | http://www.italianlp.it/resources/terence-and-teacher/ |1109| Easy Japanese | Japanese | Available | https://www.jnlp.org/GengoHouse/snow/t15 |1110| Easy Japanese Extended | Japanese | Available | https://www.jnlp.org/GengoHouse/snow/t23 |1111| RuAdapt Encyclopedia | Russian | Available | https://github.com/Digital-Pushkin-Lab/RuAdapt |1112| RuAdapt Fairytales | Russian | Available | https://github.com/Digital-Pushkin-Lab/RuAdapt |1113| RuSimpleSentEval | Russian | Available | https://github.com/dialogue-evaluation/RuSimpleSentEval |1114| RuWikiLarge | Russian | Available | https://github.com/dialogue-evaluation/RuSimpleSentEval |1115| SloTS | Slovene | Available | https://github.com/sabina-skubic/text-simplification-slovene |1116| SimplifyUR | Urdu | Pending | https://github.com/harisbinzia/SimplifyUR |1117| PorSimples | Brazilian Portuguese | Available | [sandra@icmc.usp.br](mailto:sandra@icmc.usp.br) |1118 1119### On Request Datasets1120The authors of the original papers must be contacted for on request datasets.  Contact information for the authors of each dataset is provided below.1121| Dataset | Language | Contact |1122|---|---|---|1123| CBST | Basque | http://www.ixa.eus/node/13007?language=en <br/> [itziar.gonzalezd@ehu.eus](mailto:itziar.gonzalezd@ehu.eus) |1124| DSim | Danish | [sk@eyejustread.com](mailto:sk@eyejustread.com) |1125| Newsela EN | English | [https://newsela.com/data/](https://newsela.com/data/) |1126| Newsela ES | Spanish | [https://newsela.com/data/](https://newsela.com/data/) |1127| German News | German | [ebling@cl.uzh.ch](mailto:ebling@cl.uzh.ch) |1128| Simple German | German | [ebling@cl.uzh.ch](mailto:ebling@cl.uzh.ch) |1129| Simplext | Spanish | [horacio.saggion@upf.edu](mailto:horacio.saggion@upf.edu) |1130| RuAdapt Literature | Russian | Partially Available: https://github.com/Digital-Pushkin-Lab/RuAdapt <br/> Full Dataset: [anna.dmitrieva@helsinki.fi](mailto:anna.dmitrieva@helsinki.fi) |