datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
context_qa_sum_qwen3_synthetic
Context-based QA and Summarization Synthetic Dataset
Overview
This dataset contains synthetic context-based question-answering (QA) and summarization data. The data was synthesized using:
Source context: openbmb/Ultra-FineWeb
Synthesis model: Qwen3-30B-A3B-Instruct-2507
Each context is obtained by taking the initial segment of raw pretraining text from Ultra-FineWeb, truncated to at most the corresponding number of tokens, while ensuring the truncation does not occur in… See the full description on the dataset page: https://huggingface.co/datasets/yuyijiong/context_qa_sum_qwen3_synthetic.paracrawl_context
Dataset Card for ParaCrawl_Context
This is a dataset for document-level machine translation introduced in the ACL 2024 paper Document-Level Machine Translation with Large-Scale Public Parallel Data. It is a dataset consisting of parallel sentence pairs from the ParaCrawl dataset along with corresponding preceding context extracted from the webpages the sentences were crawled from.
Dataset Details
Dataset Description
This dataset adds document-level… See the full description on the dataset page: https://huggingface.co/datasets/Proyag/paracrawl_context.ledger-long-context-KPI-QA
LEDGER — Long-Context KPI Question Answering & Page Retrieval
This dataset is part of the LEDGER (Long-context Evaluation of Documents for
Grounded Extraction and Retrieval) benchmark.
It supports two of the three LEDGER tasks:
Page-level KPI retrieval — given a natural-language question about a financial
KPI and the corresponding annual report, retrieve the relevant page(s). Each row
includes TREC-style graded relevance judgments (qrels) over all candidate pages.… See the full description on the dataset page: https://huggingface.co/datasets/artefactory/ledger-long-context-KPI-QA.ledger-long-context-multi-kpi
the LEDGER Long-Context Multi-KPI extraction datasets and benchmarks.
OCR'd annual reports with ground-truth KPI values for financial information extraction benchmarking.
Dataset Description
This dataset pairs OCR-extracted annual report text (from DeepSeek OCR) with structured KPI ground-truth values. It is designed for evaluating LLM-based financial information extraction, retrieval, and needle-in-a-haystack tasks.
Configs
Config
Reports… See the full description on the dataset page: https://huggingface.co/datasets/artefactory/ledger-long-context-multi-kpi.the_stack_v2_python_repos_pretraining_dataset_imported_context-datasetTracebench
Tracebench
This dataset contains agent trajectories (TerminalBench + SWE-bench) with two splits:
full: 3316 trajectories (2670 terminal + 646 SWE-bench)
verified: 1000 trajectories (489 SWE-bench + 511 terminal; terminal selected by step_count>=20, has incorrect steps, error-stage ratio threshold)
Agents: mini-SWE-agent (1024), OpenHands (1242), Terminus2 (923), SWE-agent (127).
Models: Anthropic/Claude-Sonnet-4, DeepSeek/DeepSeek-V3.2, Moonshot/Kimi-K2, OpenAI/GPT-5… See the full description on the dataset page: https://huggingface.co/datasets/Contextbench/Tracebench.hle-context-baseline-deepContextBias
ContextBench
The image benchmark for ContextBias: Controlled Evaluation of Bias Persistence Under Context Shift
in Text-to-Image Models (EMNLP 2026).
Text-to-image models learn associations between concepts and visual attributes that underpin many
observed forms of stereotypical bias. ContextBias is a controlled evaluation framework that asks
whether those associations are stable or adapt when a role is placed in a different context. It varies
location and activity context… See the full description on the dataset page: https://huggingface.co/datasets/shaghayegh/ContextBias.fineweb-filter-malaysian-context
HuggingFaceFW/fineweb filter Malaysian context
What is it?
We filter the original 🍷 FineWeb dataset that consists more than 15T tokens on simple Malaysian keywords.
Total tokens for the filtered dataset is 174102784199 tokens, 174B tokens.
How we do it?
We filter rows using {'malay', 'malaysia', 'melayu', 'bursa', 'ringgit'} keywords on r5.16xlarge EC2 instance for 7 days.
We calculate total tokens using tiktoken.encoding_for_model("gpt2") on c7a.24xlarge EC2… See the full description on the dataset page: https://huggingface.co/datasets/mesolitica/fineweb-filter-malaysian-context.world-bank-indicatorsContextBench
ContextBench
This repository provides:
default: the full ContextBench table (single train split).
contextbench_verified: a 500-instance subset (single split).
Columns
The dataset uses a unified schema across sources:
instance_id: ContextBench instance id (e.g., SWE-Bench-Verified__python__...).
original_inst_id: Original benchmark instance id (e.g., astropy__astropy-14539).
source: One of Verified, Pro, Poly, Multi.
language: Programming language.
repo_url: Repository… See the full description on the dataset page: https://huggingface.co/datasets/Contextbench/ContextBench.mix-context-post-training-128k
Mix-Context Post-Training Dataset for 128K Context Extension
Overview
Mix-Context Post-Training 128K is a dataset designed specifically for post-training context window extension of pretrained LLMs.
It targets the stage after base pretraining, where a model is adapted to operate over much longer contexts (up to 128K tokens) while preserving short-context behavior. The dataset mixes short- and long-context packed sequences with a controlled length distribution to support:… See the full description on the dataset page: https://huggingface.co/datasets/ghostcc3/mix-context-post-training-128k.memory-representation-contextbench-artifacts
Memory Representation ContextBench Artifacts
Dataset Summary
This repository contains processed artifacts for the paper "Memory as a Map: Prior-Trajectory Representations for Software Engineering Agents." The artifact supports reproduction and inspection of a controlled prior-context representation experiment over SWEContextBench prior-target pairs.
The experiment renders each target under four prompt conditions: no prior context, stripped Claude Code transcript… See the full description on the dataset page: https://huggingface.co/datasets/shshwtsuthar/memory-representation-contextbench-artifacts.Agentic-Long-Context-Understanding-QA 📖 Agentic Long Context Understanding 📖
Self-Taught Agentic Long Context Understanding (Arxiv).
AgenticLU refines complex, long-context queries through self-clarifications and contextual grounding, enabling robust long-document understanding in a single pass.
Installation Requirements
This codebase is largely based on OpenRLHF and Helmet, kudos to them.
The requirements are the same
pip install openrlhf
pip install -r ./HELMET/requirements.txt… See the full description on the dataset page: https://huggingface.co/datasets/yzhuang/Agentic-Long-Context-Understanding-QA.ultrabin_clean_max_chosen_min_rejected_rationalized_truthfulnessthe_stack_v2_2M_repos_pretraining_dataset_imported_context-datasetambient-acoustic-context
Dataset Card for Ambient Acoustic Context
The Ambient Acoustic Context dataset contains 1-second segments for activities that occur in a workplace setting. Each segment is associated with speaker_id.
Dataset Details
Using Amazin Mechanical Turk, crowd workers were asked to listen to 1-second segments and choose the right label. To ensure the quality of the annotations, audio segments that did not reach majority agreement among the turkers were excluded.
Dataset… See the full description on the dataset page: https://huggingface.co/datasets/flwrlabs/ambient-acoustic-context.japanese-context-relevanceこのデータセットは抽出されたハードネガティブ、質問とテキストをさらに細かく区切ったspanとの関連度スコア、さらにリランカーのスコアが含まれており、OpenProvence などのモデル学習に利用できます。
サブセットごとに元データが異なるため、ライセンスはそれぞれの提供元に従ってください。
利用可能なサブセット一覧
重複除去の有無ごとにサブセット構成をまとめました。freq2 系は MD5 ベースの頻度フィルタでデータセット全体に同一テキストが 3 回以上出現しないよう調整しており、軽量でバランスの良い学習データが欲しい場合はこちらを推奨します。同じテキストが繰り返し登場すると context_spans_relevance が過学習しやすくなるため、剪定モデルの訓練では重複を抑えることを推奨します。
サブセット
行数 (train / val / test)
テキスト重複率
推奨
msmarco-ja
492,729 / 5,000 / 5,000
約 38.9%
msmarco-ja-freq2
260,436 / 1,000… See the full description on the dataset page: https://huggingface.co/datasets/hotchpotch/japanese-context-relevance.dolmino_wiki_rephrased_qa_with_context_concatpretrain-web-mix-long-contextNormalized documents plus aligned Dolma-2 tokens and target masks.
Size
Tokens
8,689,580,607 (8.7B)
Trainable tokens
8,689,580,607 (8.7B)
Documents
281,846
Shards
89
UTF-8 bytes
37,540,769,483
Tokenizer
allenai/dolma2-tokenizer@5292e5d6c0f4
documents.parquet - document_id, text, part_ends, part_trainable,
must_not_split. The readable payload and the mask intent.
metadata.parquet - one text-free row per document: token span, source,
stratum, sizes… See the full description on the dataset page: https://huggingface.co/datasets/placeholderlabs/pretrain-web-mix-long-context.COVID-QA-unique-context-test-10-percent-validation-10-percent
Dataset Card for "COVID-QA-unique-context-test-10-percent-validation-10-percent"
More Information needed
stage4_contextwmt-da-human-evaluation-long-context
Dataset Summary
Long-context / document-level dataset for Quality Estimation of Machine Translation.
It is an augmented variant of the sentence-level WMT DA Human Evaluation dataset.
In addition to individual sentences, it contains augmentations of 2, 4, 8, 16, and 32 sentences, among each language pair lp and domain.
The raw column represents a weighted average of scores of augmented sentences using character lengths of src and mt as weights.
The code used to apply the augmentation… See the full description on the dataset page: https://huggingface.co/datasets/ymoslem/wmt-da-human-evaluation-long-context.openwebtext-gemma-2-context-128
OpenWebTextCorpus tokenized for Gemma 2 with 128 context size
This dataset is a pre-tokenized version of the Skylion007/openwebtext dataset
using the gemma tokenizer. As such, this dataset follows the same licensing as the original openwebtext dataset.
This pre-tokenization is done as a performance optimization for using the openwebtext dataset with a Gemma model (gemma-2b, gemma-2b-it, gemma-7b, gemma-7b-it).
This dataset was created using SAELens, with the following settings:… See the full description on the dataset page: https://huggingface.co/datasets/Marlon154/openwebtext-gemma-2-context-128.ContextualIntegritySyntheticDataset
Contextual Integrity Synthetic Dataset
This repository contains the synthetic dataset introduced in the paper "Contextual Integrity in LLMs via Reasoning and Reinforcement Learning".
Paper | Code | Blog
Dataset Summary
The Contextual Integrity (CI) synthetic dataset consists of 729 examples featuring diverse contexts and information disclosure norms. It is designed to instill reasoning capabilities in LLMs regarding what information is appropriate to share while… See the full description on the dataset page: https://huggingface.co/datasets/huseyinatahaninan/ContextualIntegritySyntheticDataset.pretrain-ultra-fineweb-mix-long-contextNormalized documents plus aligned Dolma-2 tokens and target masks.
Size
Tokens
1,367,358,024 (1.4B)
Trainable tokens
1,367,358,024 (1.4B)
Documents
48,077
Shards
73
UTF-8 bytes
6,386,740,105
Tokenizer
allenai/dolma2-tokenizer@5292e5d6c0f4
documents.parquet - document_id, text, part_ends, part_trainable,
must_not_split. The readable payload and the mask intent.
metadata.parquet - one text-free row per document: token span, source,
stratum, sizes… See the full description on the dataset page: https://huggingface.co/datasets/placeholderlabs/pretrain-ultra-fineweb-mix-long-context.long_context_hin_22kultrabin_clean_max_chosen_min_rejected_rationalized_honestycontext-conditioned-molecule-transfer-v10.4.1-bbb-martins-mixed-continuous-intern
BBB_Martins context-conditioned molecule transfer V10.4.1
This release preserves its direct panels and appends training-only, post-aggregate continuous assay-evidence transfer pairs. Query values remain hidden from prompts.
Train rows: 214,362
Validation rows: 30,299
Test rows: 29,919
V10.4.1 uses only continuous non-L5 assay evidence and applies the shared center-0.6, temperature-0.1 sigmoid with half-slope probability tails.
pretrain-commits-v2-mix-long-contextNormalized documents plus aligned Dolma-2 tokens and target masks.
Size
Tokens
12,571,681,749 (12.6B)
Trainable tokens
4,460,160,435 (4.5B)
Documents
992,475
Shards
327
UTF-8 bytes
49,288,867,997
Tokenizer
allenai/dolma2-tokenizer@5292e5d6c0f4
documents.parquet - document_id, text, part_ends, part_trainable,
must_not_split. The readable payload and the mask intent.
metadata.parquet - one text-free row per document: token span, source,
stratum, sizes… See the full description on the dataset page: https://huggingface.co/datasets/placeholderlabs/pretrain-commits-v2-mix-long-context.