datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
fev_datasets
Forecast evaluation datasets
This repository contains time series datasets that can be used for evaluation of univariate & multivariate forecasting models.
The main focus of this repository is on datasets that reflect real-world forecasting scenarios, such as those involving covariates, missing values, and other practical complexities.
The datasets follow a format that is compatible with the fev package.
Data format and usage
Each dataset satisfies the following… See the full description on the dataset page: https://huggingface.co/datasets/autogluon/fev_datasets.nli_fever
Overview
The original dataset can be found here
while the Github repo is here.
This dataset has been proposed in Combining fact extraction and verification with neural semantic matching networks. This dataset has been created as a modification
of FEVER.
In the original FEVER setting, the input is a claim from Wikipedia and the expected output is a label.
However, this is different from the standard NLI formalization which is basically a pair-of-sequence to label problem.
To… See the full description on the dataset page: https://huggingface.co/datasets/pietrolesci/nli_fever.FEVER_test_top_250_only_w_correct-v2
FEVERHardNegatives
An MTEB dataset
Massive Text Embedding Benchmark
FEVER (Fact Extraction and VERification) consists of 185,445 claims generated by altering sentences extracted from Wikipedia and subsequently verified without knowledge of the sentence they were derived from. The hard negative version has been created by pooling the 250 top documents per query from BM25, e5-multilingual-large and e5-mistral-instruct.
Task category
t2t
Domains
Encyclopaedic, Written… See the full description on the dataset page: https://huggingface.co/datasets/mteb/FEVER_test_top_250_only_w_correct-v2.fever_gold_evidence
Dataset Card for fever_gold_evidence
Dataset Summary
Dataset for training classification-only fact checking with claims from the FEVER dataset.
This dataset is used in the paper "Generating Label Cohesive and Well-Formed Adversarial Claims", EMNLP 2020
The evidence is the gold evidence from the FEVER dataset for REFUTE and SUPPORT claims.
For NEI claims, we extract evidence sentences with the system in "Christopher Malon. 2018. Team Papelo: Transformer Networks at FEVER.… See the full description on the dataset page: https://huggingface.co/datasets/copenlu/fever_gold_evidence.climate-fever
ClimateFEVER
An MTEB dataset
Massive Text Embedding Benchmark
CLIMATE-FEVER is a dataset adopting the FEVER methodology that consists of 1,535 real-world claims (queries) regarding climate-change. The underlying corpus is the same as FVER.
Task category
t2t
Domains
Encyclopaedic, Written
Reference
https://www.sustainablefinance.uzh.ch/en/research/climate-fever.html
How to evaluate on this task
You can evaluate an embedding model on this dataset using… See the full description on the dataset page: https://huggingface.co/datasets/mteb/climate-fever.climate_fever
Dataset Card for ClimateFever
Dataset Summary
A dataset adopting the FEVER methodology that consists of 1,535 real-world claims regarding climate-change collected on the internet. Each claim is accompanied by five manually annotated evidence sentences retrieved from the English Wikipedia that support, refute or do not give enough information to validate the claim totalling in 7,675 claim-evidence pairs. The dataset features challenging claims that relate multiple facets… See the full description on the dataset page: https://huggingface.co/datasets/tdiggelm/climate_fever.fever
FEVER
An MTEB dataset
Massive Text Embedding Benchmark
FEVER (Fact Extraction and VERification) consists of 185,445 claims generated by altering sentences extracted from Wikipedia and subsequently verified without knowledge of the sentence they were derived from.
Task categoryt2t
Domains
Encyclopaedic, Written
Reference
https://fever.ai/
How to evaluate on this task
You can evaluate an embedding model on this dataset using the following code:
import… See the full description on the dataset page: https://huggingface.co/datasets/mteb/fever.fever
Dataset Card for BEIR Benchmark
fever is one of the datasets from the Fact Checking task within BEIR, measuring Wikipedia abstract retrieval for a given fact/statement as query.
Dataset Summary
BEIR is a heterogeneous benchmark built from 18 diverse datasets representing 9 information retrieval tasks.
Fact-checking: FEVER, Climate-FEVER, SciFact
Question-Answering: NQ, HotpotQA, FiQA-2018
Bio-Medical IR: TREC-COVID, BioASQ, NFCorpus
News Retrieval: TREC-NEWS, Robust04… See the full description on the dataset page: https://huggingface.co/datasets/BeIR/fever.climate-fever-decontaminated
climate-fever (Decontaminated)
A decontaminated version of the climate-fever dataset from the BEIR benchmark, with samples found in the mgte-en pre-training dataset removed.
Decontamination methodology
Contamination was detected using a two-pass approach against the full mgte-en dataset (484 GB, 1,235 parquet files):
Pass 1: Exact hash matching
All texts (queries and corpus documents) were normalized (lowercased, unicode NFKD, whitespace collapsed) and hashed… See the full description on the dataset page: https://huggingface.co/datasets/lightonai/climate-fever-decontaminated.climate-fever
Dataset Card for BEIR Benchmark
climate-fever is one of the datasets from the Fact Checking task within BEIR, measuring Wikipedia abstract retrieval for a given claim about climate.
Dataset Summary
BEIR is a heterogeneous benchmark built from 18 diverse datasets representing 9 information retrieval tasks.
Fact-checking: FEVER, Climate-FEVER, SciFact
Question-Answering: NQ, HotpotQA, FiQA-2018
Bio-Medical IR: TREC-COVID, BioASQ, NFCorpus
News Retrieval: TREC-NEWS… See the full description on the dataset page: https://huggingface.co/datasets/BeIR/climate-fever.fever-decontaminated
fever (Decontaminated)
A decontaminated version of the fever dataset from the BEIR benchmark, with samples found in the mgte-en pre-training dataset removed.
Decontamination methodology
Contamination was detected using a two-pass approach against the full mgte-en dataset (484 GB, 1,235 parquet files):
Pass 1: Exact hash matching
All texts (queries and corpus documents) were normalized (lowercased, unicode NFKD, whitespace collapsed) and hashed with xxHash-64.… See the full description on the dataset page: https://huggingface.co/datasets/lightonai/fever-decontaminated.fever-qrels
Dataset Card for BEIR Benchmark
Dataset Summary
BEIR is a heterogeneous benchmark that has been built from 18 diverse datasets representing 9 information retrieval tasks:
Fact-checking: FEVER, Climate-FEVER, SciFact
Question-Answering: NQ, HotpotQA, FiQA-2018
Bio-Medical IR: TREC-COVID, BioASQ, NFCorpus
News Retrieval: TREC-NEWS, Robust04
Argument Retrieval: Touche-2020, ArguAna
Duplicate Question Retrieval: Quora, CqaDupstack
Citation-Prediction: SCIDOCS
Tweet… See the full description on the dataset page: https://huggingface.co/datasets/BeIR/fever-qrels.PolyOCRInject
PolyOCRInject: an OCR Prompt Injection Dataset (Turkish + English image prompt injection)
PolyOCRInject is a small, fully reproducible image prompt injection dataset for
multimodal LLM security research: prompt-injection and benign strings rendered into
images so the attack lives in pixels, exactly where a vision-language model or an OCR
document-processing pipeline has to read it. Ships with a baseline OCR prompt injection
detector and honest, measured baseline numbers.
337… See the full description on the dataset page: https://huggingface.co/datasets/fevziegeyurtsevenler/PolyOCRInject.fever-evidence-related
Dataset Summary
This dataset is extracted from Fever dataset (https://fever.ai), pre-processed and ready to train and evaluate.
The training objective is a text classification task - given a claim and evidence, predict if evidence is related to claim.
climate-fever-generated-queries
Dataset Card for BEIR Benchmark
Dataset Summary
BEIR is a heterogeneous benchmark that has been built from 18 diverse datasets representing 9 information retrieval tasks:
Fact-checking: FEVER, Climate-FEVER, SciFact
Question-Answering: NQ, HotpotQA, FiQA-2018
Bio-Medical IR: TREC-COVID, BioASQ, NFCorpus
News Retrieval: TREC-NEWS, Robust04
Argument Retrieval: Touche-2020, ArguAna
Duplicate Question Retrieval: Quora, CqaDupstack
Citation-Prediction: SCIDOCS
Tweet… See the full description on the dataset page: https://huggingface.co/datasets/BeIR/climate-fever-generated-queries.climate-fever-qrels
Dataset Card for BEIR Benchmark
Dataset Summary
BEIR is a heterogeneous benchmark that has been built from 18 diverse datasets representing 9 information retrieval tasks:
Fact-checking: FEVER, Climate-FEVER, SciFact
Question-Answering: NQ, HotpotQA, FiQA-2018
Bio-Medical IR: TREC-COVID, BioASQ, NFCorpus
News Retrieval: TREC-NEWS, Robust04
Argument Retrieval: Touche-2020, ArguAna
Duplicate Question Retrieval: Quora, CqaDupstack
Citation-Prediction: SCIDOCS
Tweet… See the full description on the dataset page: https://huggingface.co/datasets/BeIR/climate-fever-qrels.beir-fever
FEVER — BEIR, unified schema
A normalised copy of the dataset behind the mteb task FEVER, one of the tasks of the BEIR benchmark as mteb defines it. Same queries, documents
and relevance judgements as the benchmark evaluates — reshaped into one strict schema shared by every dataset
in this collection.
Source
mteb/fever @ bea83ef9e8fb (the revision pinned in mteb)
Domain · languages
fact-checking · eng
Queries / documents / qrels (all splits)
123,142 / 5,416,568… See the full description on the dataset page: https://huggingface.co/datasets/Hyukkyu/beir-fever.fever-span-annotated
FEVER (span annotated)
This dataset consists of the test set taken from the original FEVER dataset, as tsv files with the following example format:
query: Home Alone is a film.
document: Home Alone is a 1990 American Christmas comedy film written and produced by John Hughes and directed by Chris Columbus ...
spans: [121:270]
The spans indicate on character level, which sentences in the document are relevant to the claim, and are derived from the sentence level annotations of the… See the full description on the dataset page: https://huggingface.co/datasets/jinaai/fever-span-annotated.fever_bm25_top100_kilt
Dataset Card for "fever_bm25_top100_kilt"
More Information needed
BEIR-climate-fever-interpretfever-claim-related
Dataset Summary
This dataset is extracted from Climate Fever dataset (https://www.sustainablefinance.uzh.ch/en/research/climate-fever.html), pre-processed and ready to train and evaluate.
The training objective is a text classification task - given a claim and evidence, predict if claim is related to evidence.
climate-fever-vn
ClimateFEVER-VN
An MTEB dataset
Massive Text Embedding Benchmark
A translated dataset from CLIMATE-FEVER is a dataset adopting the FEVER methodology that consists of 1,535 real-world claims regarding climate-change. The process of creating the VN-MTEB (Vietnamese Massive Text Embedding Benchmark) from English samples involves a new automated system: - The system uses large language models (LLMs), specifically Coherence's Aya model, for translation. - Applies advanced embedding models… See the full description on the dataset page: https://huggingface.co/datasets/GreenNode/climate-fever-vn.mnli_anli_fevernli_wanli_lingnli_xnli_train
Dataset Card for "mnli_anli_fevernli_wanli_lingnli_xnli_train"
Train data in a harmonized format for multiple NLI datasets.
feverPoly-FEVER
Poly-FEVER: A Multilingual Hallucination Detection Benchmark
Poly-FEVER is a controlled multilingual benchmark for factuality judgment and
hallucination detection. It extends FEVER, Climate-FEVER, and SciFact to 77,973
aligned claims in 11 languages. Each claim shares the same binary label across
languages and includes topic metadata from a 22-topic LDA model, allowing
language to vary while factual content and ground truth remain fixed.
Dataset overview… See the full description on the dataset page: https://huggingface.co/datasets/HanzhiZhang/Poly-FEVER.fever
Dataset Card for "fever"
FEVER dataset with evidence retrieved by KGAT, see my paper for more info.
Labels are converted to integers with the rule:
"SUPPORTS": 0
"NOT ENOUGH INFO": 1
"REFUTES": 2
turkish-over-refusal-set
turkish-over-refusal-set
from datasets import load_dataset
ds = load_dataset("fevziegeyurtsevenler/turkish-over-refusal-set")
An XSTest-style over-refusal evaluation for Turkish (+English): 120 matched pairs of a benign-but-scary prompt and a refuse-worthy twin sharing the same trigger word (popcorn patlat vs nose patlat; chord vur vs shoot vur; process kill/öldür vs person). 480 prompts, 10 categories.
Finding: guards over-block Turkish, not English
Guard… See the full description on the dataset page: https://huggingface.co/datasets/fevziegeyurtsevenler/turkish-over-refusal-set.fever-vn
FEVER-VN
An MTEB dataset
Massive Text Embedding Benchmark
A translated dataset from FEVER (Fact Extraction and VERification) consists of 185,445 claims generated by altering sentences extracted from Wikipedia and subsequently verified without knowledge of the sentence they were derived from. The process of creating the VN-MTEB (Vietnamese Massive Text Embedding Benchmark) from English samples involves a new automated system: - The system uses large language models (LLMs), specifically… See the full description on the dataset page: https://huggingface.co/datasets/GreenNode/fever-vn.feverous_3waybeir-climate-fever
ClimateFEVER — BEIR, unified schema
A normalised copy of the dataset behind the mteb task ClimateFEVER, one of the tasks of the BEIR benchmark as mteb defines it. Same queries, documents
and relevance judgements as the benchmark evaluates — reshaped into one strict schema shared by every dataset
in this collection.
Source
mteb/climate-fever @ 47f2ac6acb64 (the revision pinned in mteb)
Domain · languages
fact-checking · eng
Queries / documents / qrels
1,535 / 5… See the full description on the dataset page: https://huggingface.co/datasets/Hyukkyu/beir-climate-fever.