Flaglab/esnlir-human-validated
ESNLIR — human-validated subset 972 sentence pairs from ESNLIR whose label was confirmed by human annotators — the validated subset used in An Analysis of the Performance of Large Language Models in Spanish NLI Datasets with Causal Relationships (IBERAMIA 2026, to appear). Part of the ESNLIR-LLM collection, used in Pacolas/NLI-via-LLM. A random sample of 2,136 instances from the full corpus was labeled by 27 Spanish-speaking university students, one label per pair. Only pairs… See the full description on the dataset page: https://huggingface.co/datasets/Flaglab/esnlir-human-validated.
ESNLIR — human-validated subset
972 sentence pairs from ESNLIR whose label was confirmed by human annotators — the validated subset used in An Analysis of the Performance of Large Language Models in Spanish NLI Datasets with Causal Relationships (IBERAMIA 2026, to appear). Part of the **ESNLIR-LLM** collection, used in `Pacolas/NLI-via-LLM`.
A random sample of 2,136 instances from the full corpus was labeled by 27 Spanish-speaking university students, one label per pair. Only pairs whose majority-annotated label matched the original connector-derived label were kept.
Format
JSON Lines — one object per line. Same fields as `Flaglab/esnlir-test`, plus dataset_connector and `connection_type`.
The gold label is `connection_type` (the human-validated label), not connector_type. Both columns are present, which is what makes this subset useful: where they disagree, the connector-derived label did not survive human review.
Label distribution
Unbalanced, unlike the test split.
Genre distribution
Usage
from huggingface_hub import hf_hub_download
import json
p = hf_hub_download("Flaglab/esnlir-human-validated",
"labeled_final_dataset.jsonl", repo_type="dataset")
rows = [json.loads(l) for l in open(p) if l.strip()]Citation
These splits were packaged for the following paper:
@InProceedings{perez2026llmspanishnlicausal,
author = {P{\'e}rez, Nicol{\'a}s and Portela, Johan R. and Manrique, Ruben},
title = {An Analysis of the Performance of Large Language Models in Spanish
NLI Datasets with Causal Relationships},
booktitle = {Advances in Artificial Intelligence -- IBERAMIA 2026},
year = {2026},
publisher = {Springer Nature Switzerland},
address = {Cham},
note = {To appear},
}The data itself is ESNLIR, released here under CC BY 4.0 — attribution to the source corpus is a condition of that licence:
@InProceedings{portela2025esnlirspanishmultigenredataset,
author = {Portela, Johan R. and P{\'e}rez-Ter{\'a}n, Nicol{\'a}s and Manrique, Rub{\'e}n},
editor = {Florez, Hector and Peluffo-Ordo{\~{n}}ez, Diego},
title = {{ESNLIR}: Expanding Spanish {NLI} Benchmarks with Multi-genre and Causal Annotation},
booktitle = {Applied Informatics},
year = {2026},
publisher = {Springer Nature Switzerland},
address = {Cham},
pages = {345--361},
isbn = {978-3-032-07175-0},
doi = {10.1007/978-3-032-07175-0_23},
}