datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
LongBench-v2
LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks
🌐 Project Page: https://longbench2.github.io
💻 Github Repo: https://github.com/THUDM/LongBench
📚 Arxiv Paper: https://arxiv.org/abs/2412.15204
LongBench v2 is designed to assess the ability of LLMs to handle long-context problems requiring deep understanding and reasoning across real-world multitasks. LongBench v2 has the following features: (1) Length: Context length ranging from 8k to… See the full description on the dataset page: https://huggingface.co/datasets/zai-org/LongBench-v2.LongBenchLongBench is a comprehensive benchmark for multilingual and multi-task purposes, with the goal to fully measure and evaluate the ability of pre-trained language models to understand long text. This dataset consists of twenty different tasks, covering key long-text application scenarios such as multi-document QA, single-document QA, summarization, few-shot learning, synthetic tasks, and code completion.LongBenchLongBench-Pro
LongBench Pro: A More Realistic and Comprehensive Bilingual Long-Context Evaluation Benchmark
LongBench-Pro, containing 1,500 samples, is entirely built on authentic, natural long documents and includes 11 primary tasks and 25 secondary tasks, covering all long-context capabilities assessed by existing benchmarks. It employs diverse evaluation metrics, enabling a more fine-grained measurement of model abilities, and provides a balanced set of bilingual samples in both… See the full description on the dataset page: https://huggingface.co/datasets/caskcsg/LongBench-Pro.longbench-v2
Citation
@article{bai2024longbench2,
title={LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks},
author={Yushi Bai and Shangqing Tu and Jiajie Zhang and Hao Peng and Xiaozhi Wang and Xin Lv and Shulin Cao and Jiazheng Xu and Lei Hou and Yuxiao Dong and Jie Tang and Juanzi Li},
journal={arXiv preprint arXiv:2412.15204},
year={2024}
}
LongBenchlongbenchlongbenchv2_fc_labellevel_checklongbenchLongBench
Introduction
LongBench is the first benchmark for bilingual, multitask, and comprehensive assessment of long context understanding capabilities of large language models. LongBench includes different languages (Chinese and English) to provide a more comprehensive evaluation of the large models' multilingual capabilities on long contexts. In addition, LongBench is composed of six major categories and twenty one different tasks, covering key long-text application scenarios such as… See the full description on the dataset page: https://huggingface.co/datasets/jannalu/LongBench.llm-jp-longbench-JEMHop
llm-jp-longbench-JEMHopQA
llm-jp LongBench ベンチマークについて
このデータセットは，GitHub リポジトリhttps://github.com/llm-jp/llm-jp-longbenchで公開されているllm-jp LongBenchベンチマークの評価対象データセットの一部として構築されています。
llm-jp LongBench ベンチマークは，日本語大型言語モデル（LLM）のロングコンテキスト処理能力を体系的に評価することを目的としており，複数の長文コンテキスト QA データセットを含んでいます。
本データセットはその一つです。
データセット概要
本データセットは、日本語の説明可能マルチホップ質問応答データセットJEMHopQA
（Ishii et al., 2024）を基に、Wikipedia記事を付与することで構築したロングコンテキストQA評価用データセットです。
最大65… See the full description on the dataset page: https://huggingface.co/datasets/llm-jp/llm-jp-longbench-JEMHop.longbench-longctx
longbench-longctx
Long-context requests for end-to-end LLM inference benchmarking in fastkernels — Scenario B. Exercises the regimes the bulk set can't reach: long-sequence attention (incl. sparse / sliding-window / DSA), RoPE/YaRN scaling, and large-KV decode.
What it's for
64 real long documents truncated into clean prefill-length buckets from 8K to 128K, each paired with its real multiple-choice question. Prefill-dominated: it measures how kernels scale with… See the full description on the dataset page: https://huggingface.co/datasets/sfc-gh-goliaro/longbench-longctx.longbench-view
Introduction
LongBench is the first benchmark for bilingual, multitask, and comprehensive assessment of long context understanding capabilities of large language models. LongBench includes different languages (Chinese and English) to provide a more comprehensive evaluation of the large models' multilingual capabilities on long contexts. In addition, LongBench is composed of six major categories and twenty one different tasks, covering key long-text application scenarios such as… See the full description on the dataset page: https://huggingface.co/datasets/leideng/longbench-view.YunXiaoHe-LongBench-Eval
云小鹤 0.3.7 LongBench zero-shot 评测
This repository contains the public evidence for a complete 200-example LongBench v1 HotpotQA run by 云小鹤 (YunXiaoHe) 0.3.7. The release covers the evaluation result, item-level trace, usage records, figures and recomputation code. The proprietary agent implementation is outside the release.
云小鹤以 zero-shot 方式完成了全部 200 题。运行前没有针对 HotpotQA 进行专项训练、微调、示例拟合、阈值搜索或评测集优化。
Join the open technical review to inspect the scoring protocol, propose an… See the full description on the dataset page: https://huggingface.co/datasets/HanyueShen/YunXiaoHe-LongBench-Eval.llm-jp-longbench-NIILC
llm-jp-longbench-NIILC
llm-jp LongBench ベンチマークについて
このデータセットは，GitHub リポジトリhttps://github.com/llm-jp/llm-jp-longbenchで公開されているllm-jp LongBenchベンチマークの評価対象データセットの一部として構築されています。
llm-jp LongBench ベンチマークは，日本語大型言語モデル（LLM）のロングコンテキスト処理能力を体系的に評価することを目的としており，複数の長文コンテキスト QA データセットを含んでいます。
本データセットはその一つです。
データセット概要
本データセットは，日本語質問応答データセット NIILC
（Sekine, 2003）を基に，
回答が一意に定まり，かつ時間によって正解が変化しない質問のみを選別し，
それらに対応する Wikipedia 記事をコンテキストとして付与することで構築した，
ロングコンテキスト QA 評価用データセットです。… See the full description on the dataset page: https://huggingface.co/datasets/llm-jp/llm-jp-longbench-NIILC.longbench_synthetic_v4_1
LongBench Synthetic V4.1
Dataset statistics
v4.1 = lbs_v4 train+val (verbatim) + a test split absorbing every config from dac-research/extra_evals_v1 not already in v4. Five ZeroScrolls configs whose upstream corpora collide with lbs_v3.1 train/test (gov_report, qmsum, qasper, narrative_qa, musique) were dropped outright. All test rows have rubrics backfilled via the longbench_generate_rubric_for_imported.jinja template (same path as lbs_v3.1 / lbs_v4 Stage 1b).… See the full description on the dataset page: https://huggingface.co/datasets/dac-research/longbench_synthetic_v4_1.LongBenchLongBench is a comprehensive benchmark for multilingual and multi-task purposes, with the goal to fully measure and evaluate the ability of pre-trained language models to understand long text. This dataset consists of twenty different tasks, covering key long-text application scenarios such as multi-document QA, single-document QA, summarization, few-shot learning, synthetic tasks, and code completion.longbench-qkv-qwen3-fullLongBench-v2LongBench-v2-32k-CoTlongbench_synthetic_v3_1
LongBench Synthetic V3.1
Dataset statistics
Per-subset stats over uploaded samples. Unique ctx counts distinct context strings, while the context-length buckets count samples/rows. Token counts use Qwen/Qwen3-14B.
Pool
Subset
Unique ctx
Sample rows
<8K
8-16K
16-32K
>32K
Median tok
p90 tok
Max tok
eval
hotpotqa
200
200
27
110
63
0
14,982
16,947
17,578
eval
hotpotqa_e
286
286
116
139
31
0
9,575
16,434
17,322
eval
musique
200
200
3
46
151
0
16,733
17… See the full description on the dataset page: https://huggingface.co/datasets/dac-research/longbench_synthetic_v3_1.LongBench-newLongBenchLongBenchLongBench is a comprehensive benchmark for multilingual and multi-task purposes, with the goal to fully measure and evaluate the ability of pre-trained language models to understand long text. This dataset consists of twenty different tasks, covering key long-text application scenarios such as multi-document QA, single-document QA, summarization, few-shot learning, synthetic tasks, and code completion.LongBench-multilingualWIP, please don't use yet
LongBench
LongBench
Dataset Summary
LongBench is a bilingual, multitask benchmark for evaluating long-context understanding in large language models. It covers long-text application scenarios including single-document question answering, multi-document question answering, summarization, few-shot learning, synthetic long-context tasks, and code completion.
This Hugging Face dataset repository repackages locally downloaded LongBench JSONL files into a clean, typed, data-only… See the full description on the dataset page: https://huggingface.co/datasets/GinkgoQ/LongBench.longbench-v2-32kLongbench_Samples_SpecdecLongBench-eptb-longbenchwrite