datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
Meta-Llama-3-8B-KronQ-HG
Meta-Llama-3-8B — KronQ H_G (output-side gradient covariance)
Paper: arXiv:2607.07964 · Code: GitHub
Pre-computed H_G for Meta-Llama-3-8B, the output-side curvature factor used by KronQ under the K-FAC factorization H ≈ H_X ⊗ H_G. H_G is the per-sublayer empirical-Fisher gradient covariance (E[g gᵀ] over the layer output), distinct from the standard input-side Hessian H_X (built online during calibration).
Publishing this lets you reproduce KronQ quantization without the offline… See the full description on the dataset page: https://huggingface.co/datasets/donghyunli/Meta-Llama-3-8B-KronQ-HG.Roblox-luau-coding_L1
8BitStudio/Roblox-luau-coding_L1
A dataset for training and fine-tuning AI models on Roblox Luau scripting.
Covers a wide range of scripting topics from beginner to advanced.
Summary
This dataset contains 12,306 Luau code examples designed to teach AI models
how to write scripts for Roblox. Topics range from basic part manipulation
to complex datastore systems.
Dataset Structure
Data Format
Each example is a tab-separated pair of a… See the full description on the dataset page: https://huggingface.co/datasets/8BitStudio/Roblox-luau-coding_L1.qwen3-8b-aime-2009-2024-16x
AIME Reasoning Traces · Qwen3-8B
7,680 reasoning traces for 480 AIME problems, with 16 sampled responses per problem. The corpus covers AIME I and AIME II from 2009 through 2024 and includes both correct and incorrect answers.
We created this dataset for How Should Incorrect Traces Be Used in Supervised Fine-Tuning? It is the source for the paper's larger AIME experiment, which compares correct and incorrect supervision across two disjoint sets of 106 problems.
Code and… See the full description on the dataset page: https://huggingface.co/datasets/suryadv/qwen3-8b-aime-2009-2024-16x.LLaDA_8B_folio_collected_logits_dataset
LLaDA 8B FOLIO Collected Logits Dataset
This dataset contains logits collected from the GSAI-ML/LLaDA-8B-Instruct model on the training set of the FOLIO dataset. It is intended for use in latent decomposition of token dynamics using sparse autoencoders, to enable semantic interpretability in masked denoising diffusion inference, specifically for use with the LLaDA model.
Contents
For each prompt, we record the following fields:
prompt_id: unique prompt directory… See the full description on the dataset page: https://huggingface.co/datasets/NielOk/LLaDA_8B_folio_collected_logits_dataset.LFM2.5-8B-A1B-KO-CPT-DATA
LFM2.5-8B-A1B Korean CPT Data
Prepared Korean continued-pretraining data for LLM-OS-Models/LFM2.5-8B-A1B-KO-CPT-FULL.
Files
data/ko_cpt_mix_full_lfmstyle_20260627.jsonl: prepared full CPT corpus with one JSON object per line and a text field
metadata/ko_cpt_mix_full_lfmstyle_20260627.stats.json: corpus statistics
metadata/ko_cpt_mix_full_lfmstyle_20260627.stats.json.full_report.json: per-source preprocessing report
metadata/ko_cpt_sources_full_20260627.json:… See the full description on the dataset page: https://huggingface.co/datasets/LLM-OS-Models/LFM2.5-8B-A1B-KO-CPT-DATA.DeepSeek-R1-Distill-Llama-8B-lmsys-openthoughts-tokenizedThis dataset is used for training Sparse Autoencoders (SAEs) to identify reasoning features in Large Language Models (LLMs), as described in the paper I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoders.
Code for the paper is available at: https://github.com/AIRI-Institute/SAE-Reasoning
The dataset consists of tokenized text data used for training the SAEs.
dataset_info:
features:
name: tokens
sequence: int64
splits:
name:… See the full description on the dataset page: https://huggingface.co/datasets/andreuka18/DeepSeek-R1-Distill-Llama-8B-lmsys-openthoughts-tokenized.Apertus-8B-2509-microQAT-logitsThis dataset provides a small sample of TOP-K logits computed using swiss-ai/Apertus-8B-2509 on samples from Data Phase 5 of Apertus pre-training.
Format
This data represents documents packed into chuncks of 4096 tokens separated by EOS. The provided fields are as follows:
input_ids: Input tokens.
index: Positions of top-256 highest-probability next-token predictions for each token.
exp_logits: Normalized probabilities of top-256 highest-probability next-token predictions for each… See the full description on the dataset page: https://huggingface.co/datasets/daslab-testing/Apertus-8B-2509-microQAT-logits.ultrachat-regen-qwen3-8b-non-thinking
UltraChat 200k Regen Qwen3-8B Non-Thinking
中文
本仓库包含 UltraChat 200k
train_sft 经 Qwen3-8B non-thinking 模式重新生成后的清理版本。数据从 max_tokens=4096
的全量 regen 出发，对触顶样本以 max_tokens=39999 做一步补生成，并移除
context-length 超限、尾部退化，以及 39999 max new tokens 打满但没有自然结束的
无界生成样本。
文件
文件
说明
行数
data/train-00000-of-00004.parquet ... data/train-00003-of-00004.parquet
Qwen3-8B non-thinking regen 清理样本
207,652
dropped_ids.jsonl
被排除的原始 row id 和原因
213
格式… See the full description on the dataset page: https://huggingface.co/datasets/heiheiha798/ultrachat-regen-qwen3-8b-non-thinking.act_patch_llama_3.1_8b_counterfact
Training Language Models to Explain Their Own Computations
Paper | Code
This dataset contains activation patching results used for training explainer models to predict how internal interventions affect target model outputs. It was introduced in the paper "Training Language Models to Explain Their Own Computations".
Dataset Summary
The dataset covers the Activation Patching task for the Llama-3.1-8B target model, where explainer models learn to predict the effects of… See the full description on the dataset page: https://huggingface.co/datasets/Transluce/act_patch_llama_3.1_8b_counterfact.Magpie-Tanuki-8B-97k
Magpie-Tanuki-8B-97k
Magpieの手法をweblab-GENIAC/Tanuki-8B-dpo-v1.0に対して適用し作成した、97269件の日本語対話データセットです。
特に事後的なフィルタ処理は加えていないため、クオリティの低いレコードが含まれている可能性があります。ご注意ください。
Llama-3.1-8B-Instruct-Infinity-Instruct-0625
Llama-3.1-8B-Instruct-Infinity-Instruct-0625
Dataset Description
This dataset is part of the LK-Speculators collection for speculative decoding research. It contains 660K prompt-response pairs designed for training draft models that are used alongside Llama-3.1-8B-Instruct as the target model. The dataset was created by generating responses to the prompts from Infinity-Instruct-0625 with meta-llama/Llama-3.1-8B-Instruct at temperature=1.
For more details on the training… See the full description on the dataset page: https://huggingface.co/datasets/nebius/Llama-3.1-8B-Instruct-Infinity-Instruct-0625.forest-of-audits-w0-sft-qwen3-8b
Forest of Audits W0 SFT for Qwen3-8B
This dataset is a W0 off-policy supervised fine-tuning warm-start artifact for
training a Qwen3-8B smart-contract audit agent. It is intended to teach the base
model EVMBench audit task format, terminal/action conventions, evidence-seeking
audit behavior, patch/exploit artifact style, and conservative vulnerability
report writing before any true OPD phase.
It is not true OPD data. Per the OPD scout contract, true OPD data must come
from… See the full description on the dataset page: https://huggingface.co/datasets/pranay5255/forest-of-audits-w0-sft-qwen3-8b.CoDIT-Qwen3-8B
Dataset Name
🤖 Teacher Model
📂 Dataset Link
CoDIT-Gemma3 💎
google/gemma-3-27b-it
CoDIT-Gemma3 ↗
CoDIT-Qwen3-8B 🐉
Qwen/Qwen3-8B
CoDIT-Qwen3-8B ↗
CoDIT-Qwen3-30B 🚀
Qwen/Qwen3-30B-A3B
CoDIT-Qwen3-30B ↗
CoDIT-Qwen3-8B
CoDIT-Qwen3-8B is a synthetic conversation dataset derived from LMSYS-Chat-1M [Zhang+, ICLR24].
250,333 user instructions sourced from LMSYS-Chat-1M
250,333 assistant responses automatically synthesized using CoDIT with Qwen/Qwen3-8B(No Thinking)… See the full description on the dataset page: https://huggingface.co/datasets/Tatsuya-Ichinose/CoDIT-Qwen3-8B.perfectblend-regen-qwen3-8b-non-thinking
PerfectBlend Regen Qwen3-8B Non-Thinking 32768 Clean
中文
本仓库包含 PerfectBlend 数据经 Qwen3-8B non-thinking 模式重新生成后的清理版本。数据从 max_tokens=4096 的原始 regen 结果出发，对疑似截断样本继续做 8192 / 16384 / 32768 token 补生成，并移除 context-length 超限、尾部退化，以及 32768 max new tokens 打满但没有自然结束的无界生成样本。
文件
文件
说明
行数
data/train-00000-of-00008.parquet ... data/train-00007-of-00008.parquet
Qwen3-8B non-thinking regen 清理样本
1,420,046
dropped_ids.jsonl
被排除的原始 row id 和原因
863… See the full description on the dataset page: https://huggingface.co/datasets/heiheiha798/perfectblend-regen-qwen3-8b-non-thinking.Magpie-Tanuki-8B-annotated-96k
Magpie-Tanuki-8B-annotated-96k
Magpieの手法をweblab-GENIAC/Tanuki-8B-dpo-v1.0に対して適用し作成したデータセットであるAratako/Magpie-Tanuki-8B-97kに対して、cyberagent/calm3-22b-chatを用いてinstructionに対して難易度、クオリティ、カテゴリをアノテーションしたデータセットです。
アノテーションのプロンプト
calm3によるアノテーションにはそれぞれ以下のプロンプトを利用しました。
難易度のアノテーション
# 指示
まず、与えられたユーザーの意図を特定し、その後、ユーザーのクエリの内容に基づいて難易度レベルをラベル付けしてください。
## ユーザーのクエリ
```
{input}
```
## 出力フォーマット
ユーザーのクエリに基づき、まずユーザーの意図を特定し、そのクエリを解決するために必要な知識を明示してください。
その後、難易度レベルを `very… See the full description on the dataset page: https://huggingface.co/datasets/Aratako/Magpie-Tanuki-8B-annotated-96k.act_patch_qwen3_8b_counterfact
Training Language Models to Explain Their Own Computations
This dataset is part of the work presented in the paper "Training Language Models To Explain Their Own Computations".
Code | Paper
Introduction
Language models (LMs) can learn to faithfully describe their internal computations. This dataset contains results from Activation Patching interventions on the Qwen3-8B target model using CounterFact data. Explainer models are trained on this data to predict how… See the full description on the dataset page: https://huggingface.co/datasets/Transluce/act_patch_qwen3_8b_counterfact.input_ablation_qwen3_8b_mmlu_hint
Training Language Models to Explain Their Own Computations (Input Ablations)
This dataset is part of the work presented in the paper "Training Language Models to Explain Their Own Computations".
It specifically contains data for the Input Ablations task for the Qwen3-8B target model. In this task, explainer models are trained to predict how removing "hint" tokens from an MMLU prompt with a hint changes the output of Qwen3-8B. This helps in understanding the causal relationships… See the full description on the dataset page: https://huggingface.co/datasets/Transluce/input_ablation_qwen3_8b_mmlu_hint.polaris-easy2k-qwen3-8b-speculative-sft
Speculative reasoning: SFT units from Polaris-easy2k
Places in a chain-of-thought trace where the reasoner stopped to settle something before
going on, rewritten so that the settling work runs on a concurrent branch while the main
line carries on from a guess.
emit a guess
├── branch A : do the work that settles the guess
└── branch B : carry on as if the guess held
Branch A leaves the critical path, so the saving is exactly its length.
Source traces:… See the full description on the dataset page: https://huggingface.co/datasets/yyuan244/polaris-easy2k-qwen3-8b-speculative-sft.EAGLE3-Apertus-8B-Instruct-2509-Data
EAGLE3-Apertus-8B-Instruct-2509-Data
Training dataset for the thomaskiefer/EAGLE3-Apertus-8B-Instruct-2509 speculative decoding draft model.
Dataset Description
This dataset contains ~375k multi-turn conversations used to train an Eagle3 draft model for swiss-ai/Apertus-8B-Instruct-2509.
Data Sources
The prompts are sourced from:
UltraChat - Large-scale multi-turn dialogue dataset
ShareGPT - Real user conversations
OpenThoughts-114k-math - Mathematical… See the full description on the dataset page: https://huggingface.co/datasets/thomaskiefer/EAGLE3-Apertus-8B-Instruct-2509-Data.qwen3-8b-codi-multihop-recall-data
CODI training data — multi-hop recall & pointer-chase (single-token-node reasoning)
The training data + generators + load-bearing eval code for two Qwen3-8B CODI latent-reasoning organisms:
cds-jb/qwen3-8b-codi-multihop-recall and
cds-jb/qwen3-8b-codi-pointer-chase.
Both tasks are single-token-node serial-reasoning problems: every intermediate and the final answer is a
single token (in both the Qwen3 and Gemma3 tokenizers), so each CODI latent can in principle be read with a… See the full description on the dataset page: https://huggingface.co/datasets/cds-jb/qwen3-8b-codi-multihop-recall-data.sharegpt-regen-qwen3-8b-non-thinking
ShareGPT Regen Qwen3-8B Non-Thinking 40K Clean
中文
本仓库包含 ShareGPT 数据经 Qwen3-8B non-thinking 模式重新生成、长输出重试和退化样本清理后的版本。原始 4K regen 的 max-token 命中样本依次经过 16K、40K retry；最终移除了 context length 超限、尾部退化/重复、以及人工确认的无界生成样本。
文件
文件
说明
行数
data/train-00000-of-00004.parquet
Hugging Face parquet 分片
10,944
data/train-00001-of-00004.parquet
Hugging Face parquet 分片
10,944
data/train-00002-of-00004.parquet
Hugging Face parquet 分片
10,943… See the full description on the dataset page: https://huggingface.co/datasets/heiheiha798/sharegpt-regen-qwen3-8b-non-thinking.MoS-Qwen3-8B-EAGLE3-responses
MoS — Qwen3-8B EAGLE3 Training Responses
Target-model responses for training EAGLE3 speculative-decoding draft models against
Qwen/Qwen3-8B. Built for the MoS (Mixture of
Speculators) project — a routed multi-MLP draft — and equally usable for any single-draft
EAGLE3 / SpecForge training run on Qwen3-8B.
599,087 complete assistant responses (with thinking traces) over five domains, generated
by Qwen3-8B itself so the draft learns to mimic the target's own distribution.… See the full description on the dataset page: https://huggingface.co/datasets/ryan-0608/MoS-Qwen3-8B-EAGLE3-responses.keural-cortex-8b-sft
Keural-Cortex-8B SFT dataset
The supervised fine-tuning set used to train Keural-Cortex-8B, a Korean-first
bilingual model with a 64K context window, tool calling, and hybrid
thinking/non-thinking modes.
1,568,649 rows · 1.87B estimated tokens · 1.80B real Qwen3 tokens · 73.7% Korean
Three files:
file
rows
what it is
train.jsonl
1,564,042
main set, all rows under 32,768 tokens
train_long64k.jsonl
4,607
the 32K–64K band, kept separate because it needs a different… See the full description on the dataset page: https://huggingface.co/datasets/mkd-hossain/keural-cortex-8b-sft.8B-reason-only.stride-32-test.k-8.statml-arxiv.qwen3-ids.kv-tags-explained
8B-reason-only.stride-32-test.k-8.statml-arxiv.qwen3-ids.kv-tags-explained
Tokenized, tag-wrapped form of JackHsieh/8B-reason-only.stride-32-test.k-8.statml-arxiv.qwen3-ids.
Each thought is wrapped as
<|note|>
This is a hint about a span that appears later in this document. KEY is the text immediately before that span; VALUE is a note about what might come next.
KEY: <last 8 prefix tokens>
VALUE: <thought>
<|/note|>
and stored both as text (thought_text) and as… See the full description on the dataset page: https://huggingface.co/datasets/JackHsieh/8B-reason-only.stride-32-test.k-8.statml-arxiv.qwen3-ids.kv-tags-explained.DeepSeek_0528_8B_Legal_Distill
⚖️ DeepSeek-0528-8B Legal Distill Dataset
This repository contains a high-density trajectory dataset generated during the GRPO (Group Relative Policy Optimization) training of the DeepSeek-8B architecture. It is specifically optimized for advanced Knowledge Distillation and structural legal reasoning. 🚀
💡 The Concept: "Log-as-Distillation"
Traditional training often treats logs as temporary metadata. This dataset flips that script. By capturing the multi-generation… See the full description on the dataset page: https://huggingface.co/datasets/Azzindani/DeepSeek_0528_8B_Legal_Distill.smoothie-qwen3-8b-kr-self-driving-legal-dataset-v5-cot
🇰🇷 자율주행법령 CoT 파인튜닝 데이터셋 v5
왜 이 데이터셋을 새로 만들었는가?
기존 dataset-v3 는 단순 질문-답변(Direct To Response, DTRO Style) 포맷으로 구성되어 있었습니다.
// v3 포맷 (기존)
{
"instruction": "자율주행자동차란 무엇인가요?",
"output": "자율주행자동차란 ..."
}
이 방식으로 파인튜닝한 모델(v3)을 RAG 파이프라인과 결합하여 평가한 결과, 정답률 43% 로 순정 모델(90%)에 크게 뒤처지는 것이 확인되었습니다. 실패의 핵심 원인은 다음과 같습니다:
실패 원인
설명
템플릿 과적합
모델이 논리가 아닌 답변 패턴(아닙니다 + 설명)을 암기
RAG 컨텍스트 무시
학습된 내부 패턴이 외부 검색 문서를 압도
<think> 태그 미사용
Qwen3의 추론(Chain-of-Thought) 능력이 전혀 활성화되지 않음… See the full description on the dataset page: https://huggingface.co/datasets/bluejude10/smoothie-qwen3-8b-kr-self-driving-legal-dataset-v5-cot.input_ablation_llama_3.1_8b_instruct_mmlu_hint
Training Language Models to Explain Their Own Computations - Input Ablations
This dataset is part of the research presented in the paper Training Language Models to Explain Their Own Computations.
It contains data for the Input Ablations task, where explainer models are trained to predict how removing input hints affects the target model's (Llama-3.1-8B-Instruct) predictions on MMLU questions with hints. This task evaluates whether models can understand the causal relationships… See the full description on the dataset page: https://huggingface.co/datasets/Transluce/input_ablation_llama_3.1_8b_instruct_mmlu_hint.qwen3-8b-base-deepscaler-rollouts
Qwen3-8B-Base rollouts on DeepScaleR, with verifier labels
Frozen on-policy rollouts collected for a two-branch generative-critic study. One
actor, sampled once; every downstream experiment reuses this exact batch.
Generation
actor
Qwen/Qwen3-8B-Base (chat template, enable_thinking=False)
prompts
agentica-org/DeepScaleR-Preview-Dataset, 10,000 sampled (seed 0)
samples per prompt
8
temperature / top-p
0.8 / 0.95
max new tokens
6,144 (context 8… See the full description on the dataset page: https://huggingface.co/datasets/YYYYYYibo/qwen3-8b-base-deepscaler-rollouts.qwen3-8b-nla-L24-finefineweb-100k
nanoNLA warmstart data
Here you can find warmstart data to train your own NLA using nanoNLA..
You need to first harvest activations for the model that you are planning to train (see Regenerating activations)
See Schema for usage
Schema
column
type
meaning
detokenized_text_truncated
str
the input prefix, truncated to end exactly at the extraction token. Source of truth — run it through the base model to recover the activation.
activation_layer
int… See the full description on the dataset page: https://huggingface.co/datasets/ceselder/qwen3-8b-nla-L24-finefineweb-100k.openthoughts3-qwen3-8b-300k
OpenThoughts3 300K — Qwen3-8B SFT Trajectories
This dataset contains 300,000 synthetic reasoning trajectories generated by
Qwen/Qwen3-8B from prompts sampled
from
open-thoughts/OpenThoughts3-1.2M.
It was prepared for the supervised fine-tuning stage of the
Lightning OPD Qwen3-4B experiment, where
Qwen/Qwen3-4B-Base is the student and Qwen3-8B is the teacher.
Dataset construction
Setting
Value
Prompt source
open-thoughts/OpenThoughts3-1.2M, train split… See the full description on the dataset page: https://huggingface.co/datasets/oldpilluwu/openthoughts3-qwen3-8b-300k.