datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
self-alignment-for-factualityThe data was organized and utilized in Self-Alignment for Factuality: Mitigating Hallucinations in LLMs via Self-Evaluation.
If you find our data useful, please cite our work using the following reference:
@inproceedings{zhang-etal-2024-self,
title = "Self-Alignment for Factuality: Mitigating Hallucinations in {LLM}s via Self-Evaluation",
author = "Zhang, Xiaoying and
Peng, Baolin and
Tian, Ye and
Zhou, Jingyan and
Jin, Lifeng and
Song, Linfeng and… See the full description on the dataset page: https://huggingface.co/datasets/xyingzhang/self-alignment-for-factuality.xsum_factualityNeural abstractive summarization models are highly prone to hallucinate content that is unfaithful to the input
document. The popular metric such as ROUGE fails to show the severity of the problem. The dataset consists of
faithfulness and factuality annotations of abstractive summaries for the XSum dataset. We have crowdsourced 3 judgements
for each of 500 x 5 document-system pairs. This will be a valuable resource to the abstractive summarization community.news_media_bias_and_factuality
News Media Factual Reporting and Political Bias
Dataset introduced in the paper "Mapping the Media Landscape: Predicting Factual Reporting and Political Bias Through Web Interactions" published in the CLEF 2024 main conference.
Similar to the news media reliability dataset, this dataset consists of a collections of 4K new media domains names with political bias and factual reporting labels.
Columns of the dataset:
source: domain name
bias: the political bias label. Values: "left"… See the full description on the dataset page: https://huggingface.co/datasets/sergioburdisso/news_media_bias_and_factuality.event_factuality
Event Factuality (It Happened / UDS-IH2)
Source
Decomp “It Happened” (UDS-IH2): https://decomp.io/projects/factuality/
UD English-EWT v1.2 (r1.2) for sentence reconstruction: https://github.com/UniversalDependencies/UD_English-EWT/tree/r1.2
Contains raw web/news text; included for research purposes only (no endorsement).
Task
Binary predicate-level event factuality (one row per predicate).
Labels
label: 0=false, 1=true
label_rule: single, agree, na_other, tie_conf4_vs0… See the full description on the dataset page: https://huggingface.co/datasets/compling/event_factuality.OpenDataGen-factuality-en-v0.1This synthetic dataset was generated using the Open DataGen Python library. (https://github.com/thoddnn/open-datagen)
Methodology:
Retrieve random article content from the HuggingFace Wikipedia English dataset.
Construct a Chain of Thought (CoT) to generate a Multiple Choice Question (MCQ).
Utilize a Large Language Model (LLM) to score the results then filter it.
All these steps are prompted in the 'template.json' file located in the specified code folder.
Code:… See the full description on the dataset page: https://huggingface.co/datasets/thoddnn/OpenDataGen-factuality-en-v0.1.dpo-mix5-Llama3-Factualityfactuality-rmbench-style
Factuality RM-Bench Style
Factuality RM-Bench Style is a controlled English dataset for studying whether
reward models and representation probes prefer stylistic presentation over
factual correctness. Each row contains one question, a localized correct and
incorrect proposition, and six responses formed by crossing correctness with
three presentation styles: concise, normal, and Markdown.
This repository is an export package for
factuality_rmbench_style_v6. The published data… See the full description on the dataset page: https://huggingface.co/datasets/Yunnnuy/factuality-rmbench-style.dpo-mix5-Llama3-Factuality-MinChosen9-MinDelta6Factuality_Alignment
Factual Preference Alignment Dataset
**⚠️ Warning:**This dataset contains hallucinated and synthetic responses
intentionally generated for research on robust factuality alignment.
Responses may include fabricated or incorrect information by design
to support the evaluation of hallucination-aware learning.
Dataset Summary
The AIXpert Preference Alignment Dataset is a curated collection of
45,000 factuality-aware preference pairs designed to support
research on Modified… See the full description on the dataset page: https://huggingface.co/datasets/vector-institute/Factuality_Alignment.squad_v2_factuality_v1
squad_v2_factuality_v1
This dataset is derived from "squad_v2" training "context" with the following steps.
NER is run to extract entities.
Lexicon of person's name, date, organisation name and location are collected.
20% of the time, one of the text attribute (person's name, date, organisation name and location) is randomly replaced. For consistency of context, all other place with the same name is also replaced.
Purpose of the Dataset
The purpose of this dataset… See the full description on the dataset page: https://huggingface.co/datasets/kenhktsui/squad_v2_factuality_v1.nvidia_NVLM-D-72B-jdgfct-Factualitydpo-qwen2572b-athene70b-jdg-Llama3-Factualitydpo-Llama31-70b-NVLM-72b-Llama3-Factualityfactuality-benchmark-preview
AIgentic Factuality Benchmark — Preview
This repository is a placeholder for AIgentic’s upcoming open benchmark for evaluating factuality and hallucination control in enterprise AI systems.
🧭 Purpose
Our goal is to set a new reliability standard for AI models deployed in high-stakes professional domains — such as law, finance, and consulting.
🔬 Coming Soon
Example benchmark dataset (legal factuality)
Architecture overview diagram
LLM-as-a-Judge evaluation… See the full description on the dataset page: https://huggingface.co/datasets/AIgenticLLC/factuality-benchmark-preview.squad_v2_factuality_v2
Dataset Card for "squad_v2_factuality_v2"
More Information needed
dpo-nemotron70b-nvlm72b-jdg-Llama3-Factualityfactuality-prediction-dataset
Introduction
Factuality classification/quantification is one of the most difficult tasks in NLP.
As apposed to sentiment analysis or other NLP tasks with statistical patterns, this task requires syntactic dependency patterns (aka, paradigmatics).
In fact, N. Jiang et al have demonstrated BERTs inability to recognize paradigmatics.
Dataset Description
This dataset was constructed by H. Kilicoglu et al to predict the factuality expressed in text about a certain… See the full description on the dataset page: https://huggingface.co/datasets/adlbh/factuality-prediction-dataset.meta-llama_Llama-3.1-70B-Instruct-jdgfct-Factualitysquad_v2_factuality_v2_top5
Dataset Card for "squad_v2_factuality_v2_top5"
More Information needed
dpo-qwen2572b-llama3170b-jdg-Llama3-Factualitymeta-llama_Llama-3.1-8B-Instruct-jdgfct-Factualitydpo-Llama31-70b-NVLM-72b-Llama3-FactualityQwen_Qwen2-7B-Instruct-jdgfct-Factualitytrain_akimbio_apertus_8b_factuality_texts_cleanedtrain_akimbio_apertus_8b_factuality_gpt-5.1_annotated_3Nexusflow_Athene-70B-jdgfct-Factualitytrain_akimbio_apertus_8b_factuality_textsfactuality-v0