datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
imatrix-calibration
Importance Matrix Calibration Datasets
This repository provides calibration datasets used to generate importance matrices (imatrix), which are required to minimize errors when quantizing models with LLaMA C++.
The llama-imatrix program cannot handle parquet files directly and thus requires them to be converted into text format first. There are many ways to do this but a simple approach is to use DuckDB with the following command: duckdb -noheader -ascii -c "SELECT content FROM… See the full description on the dataset page: https://huggingface.co/datasets/eaddario/imatrix-calibration.LLM_compression_calibration
LLM Compression Calibration dataset
This dataset is the default calibration dataset used by Neural Magic for one-shot compression of Large Language Models (LLMs).
Note: This dataset is the result of active research and subject to change without notice.
Dataset Details
Dataset Sources
The current version of this dataset is compiled from data from these datasets:
garage-bAInd/Open-Platypus: 10,000 samples
Data Fields
The dataset contains 2 data… See the full description on the dataset page: https://huggingface.co/datasets/neuralmagic/LLM_compression_calibration.GLM-5.3-Flash-calibration-activations-v1
GLM-5.3-Flash calibration activations v1 (BF16, natural routing)
Per-layer block-input activations of zai-org/GLM-5.3-Flash-BF16 @ b1967181 over 92x2048
tokens of the exllamav3 standard_cal_data corpus (pinned): per context, layer_NNN.attn_in
and layer_NNN.mlp_in (bf16, post-norm linear inputs; mlp_in is the router + expert gate/up
input) and layer_NNN.router_logits (fp32, natural top-8 routing ground truth).
Per-expert Hessians E[xx^T], routing statistics and down-proj inputs… See the full description on the dataset page: https://huggingface.co/datasets/malaiwah/GLM-5.3-Flash-calibration-activations-v1.calibration-scorecards
Prediction Market Calibration Scorecards
Monthly Brier + log-loss calibration breakdowns for Kalshi + Polymarket. Each month provides mean Brier, mean log-loss, per-venue and per-category breakdowns, and a 10-bucket calibration histogram (actual vs predicted). Published with a 14-day delay after month-end to capture late resolutions.
License and Use
This dataset is released under Creative Commons Attribution 4.0 International
(CC-BY-4.0;… See the full description on the dataset page: https://huggingface.co/datasets/SimpleFunctions/calibration-scorecards.pa-warm-start-sft-xl-calibrationcalibrationmcqa_calibration_datasetInkling-Small-Multimodal-Calibration
Inkling-Small Multimodal Calibration
The exact 1,663 samples used for BF16 routed-expert importance collection
for Inkling-Small Mixed Quant GGUF.
This is calibration material, not a held-out evaluation benchmark.
The primary balanced pass is:
Category
Samples
Valid decoder tokens
Share
Text / reasoning
462
471,858
44.976%
Code / tool-oriented source text
205
209,715
19.989%
Real image / document
486
262,476
25.018%
Real speech audio
309
105,080
10.016%
Total… See the full description on the dataset page: https://huggingface.co/datasets/Baekpica/Inkling-Small-Multimodal-Calibration.calibrationdataset.txt: https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8 + skt/kobest_v1 (boolq train)
dataset2.txt: https://gist.github.com/tristandruyen/9e207a95c7d75ddf37525d353e00659c + skt/kobest_v1 (boolq train)
safety-calibration-cases
Safety Calibration Cases
Rights & intended use: legacy public research corpus / portfolio
artifact. Hosted frontier-model outputs are research-only inputs under
project policy (synthetic-factory#161):
intended_use: research_only, project_training_policy: blocked. Not
training data for any model-weight update. Machine-readable record:
rights.json.
Release status: The raw, uncurated payload is now published under
data/raw/. It is available for inspection and reproducibility… See the full description on the dataset page: https://huggingface.co/datasets/rmems/safety-calibration-cases.calibration-mixchai1-calibration-runcalibrationagentic-vbench-calibration-trajectories
AgenticVBench volleyball calibration trajectories
Native raw agent traces from the calibration of two AgenticVBench understanding
tasks, published so a reviewer can audit turn counts, prompt parity and the
no-lookup rule independently rather than taking a summary on trust.
usc-wsu-2023-volleyball-block-timeline — 23 block points, two attributions each
byu-wsu-2023-volleyball-block-timeline — 18 block points, three attributions each
The tasks themselves, the answer keys, the… See the full description on the dataset page: https://huggingface.co/datasets/gavinlaw/agentic-vbench-calibration-trajectories.DeepSeek-R1-Distill-Qwen-1.5B-Self-CalibrationThis dataset contains data for the paper Efficient Test-Time Scaling via Self-Calibration.
We propose an efficient test-time scaling method by using model confidence for dynamically sampling adjustment, since confidence can be seen as an intrinsic measure that directly reflects model uncertainty on different tasks. For example, we can incorporate the model’s confidence into self-consistency by assigning each sampled response $y_i$ a confidence score $c_i$. Instead of treating all responses… See the full description on the dataset page: https://huggingface.co/datasets/HINT-lab/DeepSeek-R1-Distill-Qwen-1.5B-Self-Calibration.ptq-calibration-corpus
PTQ Calibration Corpus
A compact, high-density corpus built specifically for post-training
quantization of chat, coding, and agentic language models.
This is not random web text and it is not a repackaged benchmark. The corpus
was deliberately assembled to exercise varied scientific and technical prose,
numeric structure, multilingual code, tool schemas, long debugging
trajectories, CUDA/Triton vocabulary, correctness recovery, and
architecture-sensitive optimization.
This is… See the full description on the dataset page: https://huggingface.co/datasets/RESMP-DEV/ptq-calibration-corpus.korean-imatrix-calibration-corpus
Korean imatrix Calibration Corpus — KO-i1 보정 코퍼스
한국어 중심 imatrix 보정 코퍼스의 첫 공개 릴리스 (우리가 아는 한).
공개 GGUF 양자화 생태계의 importance matrix는 거의 전부 영어 위주 코퍼스로
수집됩니다. 그 결과 한국어 토큰 분포에서의 양자화 손실이 체계적으로 커집니다.
이 데이터셋은 그 공백을 메우기 위해 만들어졌고, 실측으로 효과가 입증됐습니다.
실측 효과 (이 코퍼스로 만든 KO-i1 릴리스들)
릴리스
비교 대상
결과
kanana-1.5-8b KO-i1
영어 보정 i1
저비트 KLD -5~6% (IQ2_M 3.3σ), 비트 낮을수록 이득 증가
Qwen3.6-35B-A3B KO-i1
영어 보정 i1
전 타입 우세, -5.1~-6.8% (최대 4.3σ), MoE는 4비트도 유의
Qwen3.8-27B-abl KO-i1
정적 양자… See the full description on the dataset page: https://huggingface.co/datasets/augustine223/korean-imatrix-calibration-corpus.Llama_3.1-8B-Instruct-Self-CalibrationThe official repository which contains the code and pre-trained models/datasets for our paper Efficient Test-Time Scaling via Self-Calibration.
🔥 Updates
[2025-3-3]: We released our paper.
[2025-2-25]: We released our codes, models and datasets.
🏴 Overview
We propose an efficient test-time scaling method by using model confidence for dynamically sampling adjustment, since confidence can be seen as an intrinsic measure that directly reflects model… See the full description on the dataset page: https://huggingface.co/datasets/HINT-lab/Llama_3.1-8B-Instruct-Self-Calibration.Qwen2.5-7B-Instruct-Self-Calibration
Efficient Test-Time Scaling via Self-Calibration
This repository contains datasets used in the paper Efficient Test-Time Scaling via Self-Calibration. The datasets are used to evaluate the effectiveness of test-time scaling methods for LLMs. Each config_name in the metadata refers to a different reasoning dataset. More detailed descriptions of each dataset are needed. Consider adding a section for each config_name with a description, statistics, and any other relevant… See the full description on the dataset page: https://huggingface.co/datasets/HINT-lab/Qwen2.5-7B-Instruct-Self-Calibration.linguistic_calibrationThis Datasets repo contains training and evaluation datasets for the paper "Linguistic Calibration of Long-Form Generations".
Please refer to our GitHub repo at https://github.com/tatsu-lab/linguistic_calibration for more information, and check out our paper for our research findings: https://arxiv.org/abs/2404.00474
pile-calibration-v5
Pile Calibration v5
A high-quality calibration dataset for LLM quantization, combining NeelNanda/pile-10k with bartowski's v5 imatrix calibration data.
Dataset Summary
Metric
Value
Total samples
11,674
Pile-10k samples
9,599
Bartowski v5 samples
2,075
Languages
English (primary), German, Spanish, French, Italian, Swedish, Russian, Arabic, Chinese, Japanese, Hindi, Dutch, Portuguese
What's Included
This dataset merges two sources:… See the full description on the dataset page: https://huggingface.co/datasets/lemon07r/pile-calibration-v5.llm-forecast-calibration
LLM Forecast Calibration Study — GLM-5.3 on resolved Manifold Markets questions
Raw generation data for the study "Does sampling K times beat thinking harder?
A controlled study of LLM forecast calibration on resolved binary questions."
Source repo: EzraStone/llm-forecast-calibration.
Data mirrored from GitHub commit 0f12f71a2c2ec8c54cafeb4231fecb87e705e660.
All eight JSONL files match the source data byte for byte. The source repository
remains canonical for analysis code… See the full description on the dataset page: https://huggingface.co/datasets/ezra77/llm-forecast-calibration.reaper-calibration
REAPER Calibration Dataset
Multi-scale calibration dataset for GRPO-based pruning of DeepSeek V4 Pro,
generated by the REAPER pipeline.
100 % real data — no synthetic samples at any scale.
Configs
Config
Samples
Domain Split
Purpose
seed-10k
10 000
50 % math, 30 % code, 20 % agentic
Quick proto / pipeline validation
specialist-300k
300 000
50 % math, 30 % code, 20 % agentic
Per-domain specialist training
production-800k
800 000
50 % math, 30 % code… See the full description on the dataset page: https://huggingface.co/datasets/keypa/reaper-calibration.jev-calibration-statistics
Confidence statistics for Jev and a self-judging Gemma 4 E2B
Aggregate statistics on the confidence scores from two judges in a retrieval benchmark: TypeSafe's Jev, pinned to jev-1.13.0, and Gemma 4 E2B judging its own work.
End to end, the pipeline with Jev making every decision did not beat the same pipeline with no judge: it scored 0.612 against 0.740, missed its main pre-registered bar, made about the same number of mistakes on questions both answered, and lost because it… See the full description on the dataset page: https://huggingface.co/datasets/clduab11/jev-calibration-statistics.doc_calibration_datasetqwen3_dwq_calibration_1332_235b
Qwen3 DWQ Calibration Dataset (235B, 1332 samples)
This dataset contains 1,332 samples for calibrating dynamic weight quantization (DWQ) of Qwen3-235B models. It is created following the methodology of mlx-community/qwen3_dwq_calibration_1332 but using the larger Qwen3-235B model.
Dataset Description
This dataset is derived from allenai/tulu-3-sft-mixture and consists of:
610 samples processed through Qwen3-235B with explicit reasoning
722 original samples from the… See the full description on the dataset page: https://huggingface.co/datasets/mlx-community/qwen3_dwq_calibration_1332_235b.eedi-math-calibrationeval-llama-3_1-8b-cpt-calibration
CPT calibration eval results
Per-window memorization eval results for every model state we trained in the
CPT-injection calibration experiment. Each subdirectory is one (model, dataset)
pair; results.parquet has one row per song with the standard cleanslate
sliding-window schema (coverage, mean_p_z, windows, content_id, ...).
See docs/calibration_experiment/ in the CleanSlate codebase for the full
experiment description.
Naming convention
eval_run10 — Llama-CPT… See the full description on the dataset page: https://huggingface.co/datasets/arpandeepk/eval-llama-3_1-8b-cpt-calibration.K-EXAONE-236B-REAP-calibration-mix
K-EXAONE-236B REAP/NVFP4 Calibration Mix
LGAI-EXAONE/K-EXAONE-236B-A23B의 expert pruning(REAP)과 NVFP4 양자화 calibration을 위해 제작한 믹스.
총 16,780 샘플 / 101,157,434 토큰 (K-EXAONE 토크나이저 기준).
제작 목적
MoE 모델을 one-shot pruning/양자화하면 reasoning 무한 반복(한국어/영어 공통)이 발생하는 문제가 있어,
이를 방지하기 위해 아래 원칙으로 설계:
Context length 다각화: 16 토큰 ~ 245K 토큰 (짧은 지시 → 32K agentic 궤적 → 128K 장문 → 245K needle 스트레스)
한국어 대량 포함 (instruction/reasoning/tool-calling) — K-EXAONE 특화 expert 보호
reasoning trace 원형 보존 —… See the full description on the dataset page: https://huggingface.co/datasets/Baekpica/K-EXAONE-236B-REAP-calibration-mix.night-sky-calibration-ledger
Night-sky Calibration Ledger
Approved photometric calibration observations for the observatory archive.
Retained calibrations: 5
Cover observation: OBS-10004 — Zenith / R
Observation span: 2025-01-20T19:45:00Z to 2025-07-04T04:04:00Z
Instrument counts (Aurora/Meridian/Zenith): 1/2/2
Mean zero points (Aurora/Meridian/Zenith): 24.205/25.313/24.250