datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
proofwriter
Dataset Card for "proofwriter"
More Information needed
esci
Dataset Card for "esci"
ESCI product search dataset
https://github.com/amazon-science/esci-data/
Preprocessings:
-joined the two relevant files
-product_text aggregate all product text
-mapped esci_label to full name
@article{reddy2022shopping,
title={Shopping Queries Dataset: A Large-Scale {ESCI} Benchmark for Improving Product Search},
author={Chandan K. Reddy and Lluís Màrquez and Fran Valero and Nikhil Rao and Hugo Zaragoza and Sambaran Bandyopadhyay and Arnab Biswas and Anlu… See the full description on the dataset page: https://huggingface.co/datasets/tasksource/esci.logical-entailmenthttps://github.com/google-deepmind/logical-entailment-dataset
@inproceedings{
evans2018can,
title={Can Neural Networks Understand Logical Entailment?},
author={Richard Evans and David Saxton and David Amos and Pushmeet Kohli and Edward Grefenstette},
booktitle={International Conference on Learning Representations},
year={2018},
url={https://openreview.net/forum?id=SkZxCk-0Z},
}
QuALITY
Dataset Card for "QuALITY"
@article{bowman2022quality,
title={QuALITY: Question Answering with Long Input Texts, Yes!},
author={Bowman, Samuel R and Chen, Angelica and He, He and Joshi, Nitish and Ma, Johnny and Nangia, Nikita and Padmakumar, Vishakh and Pang, Richard Yuanzhe and Parrish, Alicia and Phang, Jason and others},
journal={NAACL 2022},
year={2022}
}
chaos-mnli-ambiguity
chaos-mnli-ambiguity
ChaosNLI, MNLI portion: 1,599 MNLI pairs relabeled by 100 annotators each (Nie et al., 2020).
label_dist and label_count follow the entailment/neutral/contradiction order, and gini is the Gini
coefficient of label_dist (0 = annotators evenly split, 1 = unanimous). Built from the jsonl first uploaded
here, which flattens the ChaosNLI release (https://github.com/easonnie/ChaosNLI) and adds gini; the
variable-key label_counter (a duplicate of label_count) is… See the full description on the dataset page: https://huggingface.co/datasets/tasksource/chaos-mnli-ambiguity.planbench
Dataset Card for "planbench"
https://arxiv.org/abs/2206.10498
@article{valmeekam2024planbench,
title={Planbench: An extensible benchmark for evaluating large language models on planning and reasoning about change},
author={Valmeekam, Karthik and Marquez, Matthew and Olmo, Alberto and Sreedharan, Sarath and Kambhampati, Subbarao},
journal={Advances in Neural Information Processing Systems},
volume={36},
year={2024}
}
cycic_multiplechoicehttps://colab.research.google.com/drive/16nyxZPS7-ZDFwp7tn_q72Jxyv0dzK1MP?usp=sharing
@article{Kejriwal2020DoFC,
title={Do Fine-tuned Commonsense Language Models Really Generalize?},
author={Mayank Kejriwal and Ke Shen},
journal={ArXiv},
year={2020},
volume={abs/2011.09159}
}
added for
@article{sileo2023tasksource,
title={tasksource: Structured Dataset Preprocessing Annotations for Frictionless Extreme Multi-Task Learning and Evaluation},
author={Sileo, Damien},
url=… See the full description on the dataset page: https://huggingface.co/datasets/tasksource/cycic_multiplechoice.goal-step-wikihowhttps://github.com/zharry29/wikihow-goal-step
@inproceedings{zhang-etal-2020-reasoning,
title = "Reasoning about Goals, Steps, and Temporal Ordering with {W}iki{H}ow",
author = "Zhang, Li and
Lyu, Qing and
Callison-Burch, Chris",
booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)",
month = nov,
year = "2020",
address = "Online",
publisher = "Association for Computational Linguistics",
url =… See the full description on the dataset page: https://huggingface.co/datasets/tasksource/goal-step-wikihow.procedural-typed-decisions
procedural-typed-decisions
Procedurally generated decision problems. Each row is one structured state
(JSON, or a table, CSV, key=value lines, or prose for the arithmetic,
retrieval, and aggregation configs) with several typed questions over that same state, following the
Jev / System One request shape: choice (pick one criterion), noul (a
number in [0, 1]; a probability or a yes/no), and score (an ordered rubric).
Every answer is computed exactly from the state by rules that… See the full description on the dataset page: https://huggingface.co/datasets/tasksource/procedural-typed-decisions.oasst1_dense_flat
Dataset Card for "oasst1_dense_flat"
OASST1 dataset
But where with retrieved parent_text, and where we only keep messages with dense annotations (all labels have 2 annotators)
from datasets import Dataset, DatasetDict
d={}
for split in ['train','validation']:
df=load_dataset("OpenAssistant/oasst1")[split].to_pandas()
m2t=df.set_index("message_id")['text'].to_dict()
df['parent_text']=df.parent_id.map(lambdax: m2t.get(x,''))
df=df[df.labels.map(lambda x:x!=None)]… See the full description on the dataset page: https://huggingface.co/datasets/tasksource/oasst1_dense_flat.omniplanhumicroedit
humicroedit
Humicroedit (SemEval-2020 task 7): news headlines edited to be funny.
subtask-1 grades one edited headline (meanGrade averages five 0-3 funniness grades); headline and
edited render the original headline and its edit from the <word/> markup. subtask-2 compares two
edits of the same headline (label 1 or 2 is the funnier one, 0 a tie).
Original data: SemEvalWorkshop/humicroedit. Repackaged as parquet for tasksource by scripts/upload_repackaged.py.
help-desk-tickets
Help Desk Tickets
Processed tables derived from version 3 of Mohammad Abdellatif's Mendeley Data
dataset. The source contains real helpdesk tickets and associated workflow
records from an international software company, covering January 2007 through
March 2023. Identifiers and message content were masked by the source authors
to protect privacy while retaining context.
The default config is reporter_messages, with one row per issue that has
reporter-authored text. Utterances are… See the full description on the dataset page: https://huggingface.co/datasets/tasksource/help-desk-tickets.cycic_classificationhttps://storage.googleapis.com/ai2-mosaic/public/cycic/CycIC-train-dev.zip
https://colab.research.google.com/drive/16nyxZPS7-ZDFwp7tn_q72Jxyv0dzK1MP?usp=sharing
@article{Kejriwal2020DoFC,
title={Do Fine-tuned Commonsense Language Models Really Generalize?},
author={Mayank Kejriwal and Ke Shen},
journal={ArXiv},
year={2020},
volume={abs/2011.09159}
}
added for
@article{sileo2023tasksource,
title={tasksource: Structured Dataset Preprocessing Annotations for Frictionless Extreme… See the full description on the dataset page: https://huggingface.co/datasets/tasksource/cycic_classification.imdb62https://umlt.infotech.monash.edu/?page_id=266
@article{seroussi2014authorship,
title={Authorship attribution with topic models},
author={Seroussi, Yanir and Zukerman, Ingrid and Bohnert, Fabian},
journal={Computational Linguistics},
volume={40},
number={2},
pages={269--310},
year={2014},
publisher={MIT Press One Rogers Street, Cambridge, MA 02142-1209, USA journals-info~…}
}
oasst2_dense_flat
Dataset Card for "oasst2_dense_flat"
More Information needed
lewidi
lewidi
Learning with Disagreements (LeWiDi, SemEval-2023 Task 11 and its 2025 edition): soft labels from every annotator.
One config per dataset: md_agreement (offensiveness, 5 annotators), hs_brexit (hate speech, 6), armis
(Arabic misogyny and sexism, 3), conv_abuse (abuse in user turns of chatbot dialogues, 3 or more), csc
(sarcasm rated 1-6), mp (MultiPICo irony, multilingual) and varierrnli (NLI where each annotator may accept
several labels). soft_label lists the share of… See the full description on the dataset page: https://huggingface.co/datasets/tasksource/lewidi.scifact_entailmentSciFact entailment pairs (data-only; train/validation).
LogicHaystacksethos
ethos
ETHOS multi-label: hateful comments with the share of raters who saw each aspect (violence, target, grounds).
Only the source's single split is provided.
Original data: iamollas/ethos. Repackaged as parquet for tasksource by scripts/upload_repackaged.py.
measuring-hate-speech-votes
measuring-hate-speech-votes
Measuring Hate Speech (Kennedy et al., 2020; Sachdeva et al., 2022), one row per comment with vote counts.
The source has one row per (comment, annotator). Each survey item becomes a list of vote counts over its
ordinal codes, in code order: 0-4 for the nine Likert items, where a higher code is more hateful
(sentiment: strongly positive to strongly negative; respect: strongly respectful to strongly disrespectful;
insult, humiliate, dehumanize… See the full description on the dataset page: https://huggingface.co/datasets/tasksource/measuring-hate-speech-votes.gsm8k-nli