datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
ner-eval-predictionswiki_comparable_corpus_en_de_hi_it_ko_zh
Multilingual Wikipedia Comparable Corpus (en, de, it, ko, hi, zh)
This dataset is a document-level comparable corpus of Wikipedia articles across 6 languages: English (en), German (de), Italian (it), Korean (ko), Hindi (hi), and Chinese (zh).
The key property is alignment across languages: entries are topic-matched such that, for a given index i, dataset["en"][i] is comparable to dataset["de"][i], dataset["it"][i], … (and likewise via the aligned_id field).
Dataset… See the full description on the dataset page: https://huggingface.co/datasets/impresso-project/wiki_comparable_corpus_en_de_hi_it_ko_zh.amr-true-paraphrases
True Paraphrases Test Set
The True Paraphrases sentence/phrase pairs derived from the AMR Annotation Guidelines. It was introduced as part of the PARAPHRASUS: A Comprehensive Benchmark for Evaluating Paraphrase Detection Models.
For more details, refer to the original paper that was presented at COLING 2025.
Citation
If you use this dataset, please cite it using the following BibTeX entry:
@inproceedings{michail-etal-2025-paraphrasus,
title = "{PARAPHRASUS}: A… See the full description on the dataset page: https://huggingface.co/datasets/impresso-project/amr-true-paraphrases.sts-h-paraphrase-detection
STS-Hard Test Set
The STS-Hard dataset is a paraphrase detection test set derived from the STSBenchmark dataset. It was introduced as part of the PARAPHRASUS: A Comprehensive Benchmark for Evaluating Paraphrase Detection Models. The test set includes the paraphrase label as well as individual annotation labels from two annotators:
P1: The semanticist.
P2: A student annotator.
For more details, refer to the original paper that was presented at COLING 2025.
Citation… See the full description on the dataset page: https://huggingface.co/datasets/impresso-project/sts-h-paraphrase-detection.