LianeMarilin/long-context-qa-curated-20

<!-- HFDATASETCARD_START -->

Dataset Card / 数据集卡

Dataset Description / 数据集简介

This public release contains 20 curated samples selected from a 10,000-record long-context QA collection. It targets retrieval over long documents, cross-section evidence synthesis, numerical reasoning, timeline reconstruction, and structured answer evaluation. The public subset contains 15 short-answer questions and 5 multiple-choice questions, balanced across Chinese and English.

本公开版本从 10,000 条长上下文问答数据中精选 20 条，用于长文档检索、跨段证据整合、数值推理、时序重建和结构化回答评测；其中简答题 15 道、选择题 5 道，中英文各 10 道。

• —Task: long-context question answering and LLM evaluation
• —Languages: Chinese (zh) and English (en)
• —Public size: 20 QA rows, 160 associated rollout records
• —Full collection: 10,000 records
• —Context range: approximately 8k–256k length buckets
• —Source repository: GitHub

Dataset Structure and Splits / 数据结构与划分

train is used as the Viewer container split. The 20-row public release is an evaluation sample rather than a prescribed model-training partition. Context documents, rollout traces, scores, and validation artifacts remain as individual files in the repository.

Data Fields / 字段说明

Loading / 加载方式

from datasets import load_dataset

ds = load_dataset("LianeMarilin/long-context-qa-curated-20")
print(ds["train"][0])

The JSONL is also directly readable with any line-oriented JSON parser; no archive extraction step is required.

Curation, Intended Uses, and Limitations / 筛选、用途与局限

Samples were selected for bilingual balance, answerability, evidence traceability, question-type coverage, and difficulty. Suitable uses include long-context QA evaluation, retrieval analysis, judge calibration, and error analysis. Report results separately by language, question type, domain, and context-length bucket. The public subset is small and intentionally difficult, so it should not be treated as a population-level estimate or as a complete training corpus. Source-page availability and licensing may vary; users should review each record's provenance fields before redistribution.

License and Citation / 许可与引用

Dataset metadata uses license: other: benchmark-authored annotations and scripts may be reused with attribution, while linked source material remains governed by its original terms. Cite the dataset repository and preserve record-level provenance.

@misc{long_context_qa_curated_20_2026,
  title  = {Long Context QA: Curated 20},
  author = {Liane Marilin},
  year   = {2026},
  url    = {https://huggingface.co/datasets/LianeMarilin/long-context-qa-curated-20}
}

<!-- HFDATASETCARD_END -->

<p align="center"> <img src="assets/long-context-qa-banner.png" width="100%" alt="Long Context QA Curated 20 banner" /> </p>

<h1 align="center">Long Context QA · Curated 20</h1>

<p align="center"> 面向长文档检索、跨段证据整合与结构化推理的精选问答数据集<br/> <em>A compact, inspectable benchmark for long-context question answering.</em> </p>

<p align="center"> <img alt="records" src="https://img.shields.io/badge/records-20-2563eb?style=flat-square" /> <img alt="rollouts" src="https://img.shields.io/badge/rollouts-160-7c3aed?style=flat-square" /> <img alt="languages" src="https://img.shields.io/badge/languages-ZH%20%7C%20EN-0891b2?style=flat-square" /> <img alt="validated" src="https://img.shields.io/badge/validation-passing-059669?style=flat-square" /> <img alt="jsonl" src="https://img.shields.io/badge/format-JSONL-f59e0b?style=flat-square" /> </p>

✨ 数据集亮点

📊 数据概览

<details> <summary><strong>查看领域分布</strong></summary>

</details>

🧭 评测链路

flowchart LR
    A[Long Context] --> B[Question]
    B --> C[8 Independent Rollouts]
    C --> D[Semantic Judge]
    D --> E[Per-run Scores]
    E --> F[Average Accuracy]
    A --> G[Evidence]
    G --> D

🗂️ 仓库结构

.
├── assets/                         # README视觉素材
├── configs/                        # 三批rollout配置
├── contexts/                       # 20份长上下文原文
├── data/
│   ├── long_context_qa_curated_20.jsonl
│   ├── dataset_card.json
│   └── index.csv
├── docs/
│   └── selection-report.md         # 逐题精选清单
├── rollouts/                       # 每题8轮回答、评分与裁判
├── scripts/
│   └── validate_dataset.py
└── README.md

🚀 快速开始

读取数据

import json
from pathlib import Path

path = Path("data/long_context_qa_curated_20.jsonl")
records = [json.loads(line) for line in path.read_text(encoding="utf-8").splitlines()]

print(f"records: {{len(records)}}")
print(records[0]["question"])
print(records[0]["answer"])

运行完整校验

python3 scripts/validate_dataset.py

成功时输出：

PASS · 20 records · 20 contexts · 160 rollouts

🧩 核心字段

🏅 精选策略

• —排除源包中已明确标记为证据不足或答案不唯一的题目。
• —相同context只保留一道题，降低重复度。
• —纳入源数据中全部5道有效选择题。
• —在剩余题目中平衡语言、领域、长度桶与任务类型。
• —保留原track_id与context_id，便于回溯。
• —修正000026的JSON标准答案和000037的题型元数据。

完整记录见 精选题清单。

🔍 单条记录示例

{{
  "track_id": "longqa_delivery_000001",
  "context_id": "context_000001",
  "question_type": "短答案题",
  "language": "Chinese",
  "domain": "软件与工程",
  "token_length": 8488,
  "file_path": "contexts/context_000001.txt",
  "question": "...",
  "answer": ["..."],
  "review_result": {{"status": "pass"}},
  "difficulty_result": {{"rollout_count": 8}}
}}

🛠️ 已修订记录

📌 使用说明

完整数据集共 10,000条，本仓库从中精选 20条高质量样例，用于展示数据格式、任务类型、长上下文推理难度及完整评测流程。

<p align="center"> <strong>Built for inspectable long-context evaluation.</strong><br/> Context → Evidence → Reasoning → Answer </p>