datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
imagescertificatessql-create-context
Overview
This dataset builds from WikiSQL and Spider.
There are 78,577 examples of natural language queries, SQL CREATE TABLE statements, and SQL Query answering the question using the CREATE statement as context. This dataset was built with text-to-sql LLMs in mind, intending to prevent hallucination of column and table names often seen when trained on text-to-sql datasets. The CREATE TABLE statement can often be copy and pasted from different DBMS and provides table names, column… See the full description on the dataset page: https://huggingface.co/datasets/b-mc2/sql-create-context.activating_contexts_16kcontext_qa_sum_qwen3_synthetic
Context-based QA and Summarization Synthetic Dataset
Overview
This dataset contains synthetic context-based question-answering (QA) and summarization data. The data was synthesized using:
Source context: openbmb/Ultra-FineWeb
Synthesis model: Qwen3-30B-A3B-Instruct-2507
Each context is obtained by taking the initial segment of raw pretraining text from Ultra-FineWeb, truncated to at most the corresponding number of tokens, while ensuring the truncation does not occur in… See the full description on the dataset page: https://huggingface.co/datasets/yuyijiong/context_qa_sum_qwen3_synthetic.paracrawl_context
Dataset Card for ParaCrawl_Context
This is a dataset for document-level machine translation introduced in the ACL 2024 paper Document-Level Machine Translation with Large-Scale Public Parallel Data. It is a dataset consisting of parallel sentence pairs from the ParaCrawl dataset along with corresponding preceding context extracted from the webpages the sentences were crawled from.
Dataset Details
Dataset Description
This dataset adds document-level… See the full description on the dataset page: https://huggingface.co/datasets/Proyag/paracrawl_context.ledger-long-context-KPI-QA
LEDGER — Long-Context KPI Question Answering & Page Retrieval
This dataset is part of the LEDGER (Long-context Evaluation of Documents for
Grounded Extraction and Retrieval) benchmark.
It supports two of the three LEDGER tasks:
Page-level KPI retrieval — given a natural-language question about a financial
KPI and the corresponding annual report, retrieve the relevant page(s). Each row
includes TREC-style graded relevance judgments (qrels) over all candidate pages.… See the full description on the dataset page: https://huggingface.co/datasets/artefactory/ledger-long-context-KPI-QA.ledger-long-context-multi-kpi
the LEDGER Long-Context Multi-KPI extraction datasets and benchmarks.
OCR'd annual reports with ground-truth KPI values for financial information extraction benchmarking.
Dataset Description
This dataset pairs OCR-extracted annual report text (from DeepSeek OCR) with structured KPI ground-truth values. It is designed for evaluating LLM-based financial information extraction, retrieval, and needle-in-a-haystack tasks.
Configs
Config
Reports… See the full description on the dataset page: https://huggingface.co/datasets/artefactory/ledger-long-context-multi-kpi.the_stack_v2_python_repos_pretraining_dataset_imported_context-datasetContextASR-Bench
ContextASR-Bench: A Massive Contextual Speech Recognition Benchmark
Automatic Speech Recognition (ASR) has been extensively investigated, yet prior benchmarks have largely focused on assessing the acoustic robustness of ASR models, leaving evaluations of their linguistic capabilities relatively underexplored. This largely stems from the limited parameter sizes and training corpora of conventional ASR models, leaving them with insufficient world knowledge, which is crucial for… See the full description on the dataset page: https://huggingface.co/datasets/MrSupW/ContextASR-Bench.Tracebench
Tracebench
This dataset contains agent trajectories (TerminalBench + SWE-bench) with two splits:
full: 3316 trajectories (2670 terminal + 646 SWE-bench)
verified: 1000 trajectories (489 SWE-bench + 511 terminal; terminal selected by step_count>=20, has incorrect steps, error-stage ratio threshold)
Agents: mini-SWE-agent (1024), OpenHands (1242), Terminus2 (923), SWE-agent (127).
Models: Anthropic/Claude-Sonnet-4, DeepSeek/DeepSeek-V3.2, Moonshot/Kimi-K2, OpenAI/GPT-5… See the full description on the dataset page: https://huggingface.co/datasets/Contextbench/Tracebench.long-context-text-summarization-alpaca-formatpersona-drift-contextecho
ContextEcho — Released Dataset
Per-cell evaluation corpus and donated session prefixes for the ContextEcho
benchmark. This Hugging Face repository hosts the released dataset artifacts.
The canonical project page, latest README, code, reproduction instructions, and
donation workflow are maintained on GitHub:
https://github.com/Accenture/ContextEcho
Donate a coding-agent session: https://accenture.github.io/ContextEcho/donate/
For the formal datasheet, see DATASHEET.md.… See the full description on the dataset page: https://huggingface.co/datasets/contextecho2026/persona-drift-contextecho.activating_contexts_131k_layers_21_42contextual_testCheck out the paper.
ContextBias
ContextBench
The image benchmark for ContextBias: Controlled Evaluation of Bias Persistence Under Context Shift
in Text-to-Image Models (EMNLP 2026).
Text-to-image models learn associations between concepts and visual attributes that underpin many
observed forms of stereotypical bias. ContextBias is a controlled evaluation framework that asks
whether those associations are stable or adapt when a role is placed in a different context. It varies
location and activity context… See the full description on the dataset page: https://huggingface.co/datasets/shaghayegh/ContextBias.scientific-figures-captions-context
Dataset Card for Scientific Figures, Captions, and Context
A novel vision-language dataset of scientific figures taken directly from research papers.
We scraped approximately ~150k papers, with about ~690k figures total. We extracted each figure's caption and label from the paper. In addition, we searched through each paper to find references of each figure and included the surrounding text as 'context' for this figure.
All figures were taken from arXiv research papers.… See the full description on the dataset page: https://huggingface.co/datasets/mawadalla/scientific-figures-captions-context.Multi-turn_Long-context_Benchmark_for_LLMs
LoopServe: An Adaptive Dual-phase LLM Inference Acceleration System for Multi-Turn Dialogues
Arxiv: https://www.arxiv.org/abs/2507.13681
Huggingface: https://huggingface.co/papers/2507.13681
Introduction
LoopServe Multi-Turn Dialogue Benchmark is a comprehensive evaluation dataset comprising multiple diverse datasets designed to assess large language model performance in realistic conversational scenarios.
Unlike traditional benchmarks that place queries only at the end… See the full description on the dataset page: https://huggingface.co/datasets/TreeAILab/Multi-turn_Long-context_Benchmark_for_LLMs.fineweb-filter-malaysian-context
HuggingFaceFW/fineweb filter Malaysian context
What is it?
We filter the original 🍷 FineWeb dataset that consists more than 15T tokens on simple Malaysian keywords.
Total tokens for the filtered dataset is 174102784199 tokens, 174B tokens.
How we do it?
We filter rows using {'malay', 'malaysia', 'melayu', 'bursa', 'ringgit'} keywords on r5.16xlarge EC2 instance for 7 days.
We calculate total tokens using tiktoken.encoding_for_model("gpt2") on c7a.24xlarge EC2… See the full description on the dataset page: https://huggingface.co/datasets/mesolitica/fineweb-filter-malaysian-context.world-bank-indicatorsX2I-in-context-learning
X2I Dataset
Project Page: https://vectorspacelab.github.io/OmniGen/
Github: https://github.com/VectorSpaceLab/OmniGen
Paper: https://arxiv.org/abs/2409.11340
Model: https://huggingface.co/Shitao/OmniGen-v1
To achieve robust multi-task processing capabilities, it is essential to train the OmniGen on large-scale and diverse datasets. However, in the field of unified image generation, a readily available dataset has yet to emerge. For this reason, we have curated a large-scale… See the full description on the dataset page: https://huggingface.co/datasets/yzwang/X2I-in-context-learning.hle-context-baseline-deeplong_context_evallong-context-qa-curated-20
Dataset Card / 数据集卡
Dataset Description / 数据集简介
This public release contains 20 curated samples selected from a 10,000-record long-context QA collection. It targets retrieval over long documents, cross-section evidence synthesis, numerical reasoning, timeline reconstruction, and structured answer evaluation. The public subset contains 15 short-answer questions and 5 multiple-choice questions, balanced across Chinese and English.
本公开版本从 10,000 条长上下文问答数据中精选 20… See the full description on the dataset page: https://huggingface.co/datasets/LianeMarilin/long-context-qa-curated-20.finewiki-pplx-embed-context-4B-512tok-stateContextBench
ContextBench
This repository provides:
default: the full ContextBench table (single train split).
contextbench_verified: a 500-instance subset (single split).
Columns
The dataset uses a unified schema across sources:
instance_id: ContextBench instance id (e.g., SWE-Bench-Verified__python__...).
original_inst_id: Original benchmark instance id (e.g., astropy__astropy-14539).
source: One of Verified, Pro, Poly, Multi.
language: Programming language.
repo_url: Repository… See the full description on the dataset page: https://huggingface.co/datasets/Contextbench/ContextBench.Context-as-Memory-Dataset
Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrieval
SIGGRAPH Asia 2025
[Project page]
[ArXiv]
[Dataset]
File Structure
To prepare the dataset for use, merge the parts into a single zip file using the following command:
cat Context-as-Memory-Dataset_* > Context-as-Memory-Dataset.zip
After extracting Context-as-Memory-Dataset.zip, the dataset will be organized as follows:
Context-as-Memory-Dataset
├──… See the full description on the dataset page: https://huggingface.co/datasets/KlingTeam/Context-as-Memory-Dataset.memory-representation-contextbench-artifacts
Memory Representation ContextBench Artifacts
Dataset Summary
This repository contains processed artifacts for the paper "Memory as a Map: Prior-Trajectory Representations for Software Engineering Agents." The artifact supports reproduction and inspection of a controlled prior-context representation experiment over SWEContextBench prior-target pairs.
The experiment renders each target under four prompt conditions: no prior context, stripped Claude Code transcript… See the full description on the dataset page: https://huggingface.co/datasets/shshwtsuthar/memory-representation-contextbench-artifacts.Agentic-Long-Context-Understanding-QA 📖 Agentic Long Context Understanding 📖
Self-Taught Agentic Long Context Understanding (Arxiv).
AgenticLU refines complex, long-context queries through self-clarifications and contextual grounding, enabling robust long-document understanding in a single pass.
Installation Requirements
This codebase is largely based on OpenRLHF and Helmet, kudos to them.
The requirements are the same
pip install openrlhf
pip install -r ./HELMET/requirements.txt… See the full description on the dataset page: https://huggingface.co/datasets/yzhuang/Agentic-Long-Context-Understanding-QA.the_stack_v2_2M_repos_pretraining_dataset_imported_context-dataset