Lidor-Mashiach/mnli-paraphrase-bank

MNLI Paraphrase Bank

Overview

This repository contains paraphrases for hypotheses in the MultiNLI training split.

Each original hypothesis has up to five paraphrases. Every retained paraphrase passed two automated checks. The first check tested semantic equivalence with the original hypothesis. The second check tested whether the relation between the premise and the paraphrase matched the original gold label.

The dataset contains 1,923,315 paraphrase rows from 390,008 retained hypotheses.

Files

How the dataset was created

The generation pool contained 392,702 hypotheses from the MultiNLI training split.

For each hypothesis, the paraphrase generator produced 12 candidates in each round. The generator was humarin/chatgpt paraphraser on T5 base.

A candidate was kept only when all of the following conditions were met:

1. 1.Its word count was between 0.6 and 1.5 times the word count of the original hypothesis.
2. 2.The original hypothesis and the candidate were predicted to entail one another.
3. 3.The predicted NLI relation between the premise and the candidate matched the original gold label.
4. 4.The candidate was not a duplicate of an earlier candidate for the same hypothesis.

The verifier was MoritzLaurer/DeBERTa-v3-large-mnli-fever-anli-ling-wanli.

The target was five accepted paraphrases per hypothesis. A hypothesis with one to four accepted paraphrases was kept with the accepted results. A hypothesis with no accepted paraphrases was left out of the paraphrase bank.

Generation continued for up to six rounds.

Statistics

The full histograms and rejection counts appear in mnli-train-stats.json.

Suggested uses

This dataset can be used to study consistency under meaning preserving changes, NLI robustness, paraphrase based evaluation, data augmentation, and differences across MultiNLI genres.

Results should report the exact repository version, any extra filtering, the evaluated subset, the model settings, and the method used to combine paraphrase level results.

Limitations

The paraphrases were generated and checked by models. They were not reviewed one by one by human annotators.

Automated checks can accept an imperfect paraphrase or reject a valid one.

The dataset covers the MultiNLI training split only. It does not replace the official development or test sets.

Results may depend on the generator and verifier used to build the dataset.

License

MultiNLI contains material from several sources. The original dataset card lists CC BY 3.0, CC BY-SA 3.0, MIT, and other terms.

This repository keeps the same mixed license notice. Users should review the original MultiNLI documentation and follow the terms that apply to the relevant source material.

Associated research

This dataset was created as part of the following research manuscript:

“Opening the Black Box: Localizing semantic inconsistency in NLI models with Deep k -Nearest Neighbors”

The manuscript is in preparation. It has not been submitted or published.

This section will be updated when a public preprint or an accepted version becomes available.

Citation

Until the paper is public, please cite this dataset repository:

@misc{mashiach2026mnliparaphrasebank,
  author = {Lidor Mashiach},
  title = {MNLI Paraphrase Bank},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/datasets/Lidor-Mashiach/mnli-paraphrase-bank}
}

Please also cite the original MultiNLI paper:

Adina Williams, Nikita Nangia, and Samuel R. Bowman. A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference. NAACL-HLT 2018.

Contact

Questions, corrections, and reproducibility reports can be posted in the Community tab of this repository.