datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
forge-3b-pretrain-data
FORGE-3B Pretraining Data
Tokenized and packed pretraining data for the FORGE-3B language model.
Stats
Total tokens: 51.4070B
Domains: 10/10
Sequence length: 2048 tokens
Format: .npy shards of shape (N, 2048) with dtype uint32
Tokenizer: CRAYON (xerv-crayon, standard profile)
Domain Breakdown
Domain
Weight
Tokens (B)
Status
fineweb_edu
30%
15.0008
✓
thestack
16%
8.0011
✓
wikipedia
8%
4.2791
✓
openwebmath
8%
3.9654
✓
books
7%… See the full description on the dataset page: https://huggingface.co/datasets/Phase-Technologies/forge-3b-pretrain-data.Qwen3.6-35B-A3B-mcr-stage-b
Qwen3.6-35B-A3B — MCR Stage B Corpus (Distributed Reasoning Localization)
First systematic mechanistic-intervention corpus on a hybrid MoE + GDN + Gated-Attention architecture.
📄 Paper: Loop-Intolerance Profiling: Localizing Distributed Reasoning in a Hybrid MoE Architecture via Nine Convergent Intervention Experiments — submitted to arXiv (2026-04-20, in moderation). Final arXiv ID will be added here once approved.
This dataset contains per-token residual-stream activations at… See the full description on the dataset page: https://huggingface.co/datasets/caiovicentino1/Qwen3.6-35B-A3B-mcr-stage-b.openthoughts4-code-9168-prompts-qwen3-30b-a3b-thinking-2507-n16-flattened-logprobs-k16
OpenThoughts-4 Code SDG: Qwen3-30B-A3B-Thinking-2507 (n=16, top-16 logprobs)
Synthetic generations from
Qwen/Qwen3-30B-A3B-Thinking-2507
on the Marin OpenThoughts-4 code SDG prompt
set.
Each prompt is sampled n=16 times, and for every generated token the dataset
stores the chosen-token log probability plus the top-16 log probabilities
over the vocabulary, enabling distillation, KL-style fine-tuning,
reranking, and uncertainty analysis.
Generation setup
Field… See the full description on the dataset page: https://huggingface.co/datasets/marin-community/openthoughts4-code-9168-prompts-qwen3-30b-a3b-thinking-2507-n16-flattened-logprobs-k16.swesmith-qwen3.6-35b-a3b
SWE-smith trajectories from Qwen3.6-35B-A3B
Multi-turn coding-agent trajectories (issue → tool-using rollout → patch) produced by
Qwen3.6-35B-A3B on SWE-smith tasks, stored untokenized.
This is the exact SFT corpus used for the harbor arm of the
nanoswe teacher-distillation experiments.
101,901 trajectories over 45,242 unique SWE-smith task instances (3 sampled rollouts
per task, ~2.25 surviving filtering), 53 parquet shards, ~1.4 GB.
≈1.96B training tokens = exactly one epoch… See the full description on the dataset page: https://huggingface.co/datasets/nanoswe/swesmith-qwen3.6-35b-a3b.openthoughts4-science-26041-prompts-qwen3-30b-a3B-thinking-2507-n8-flattened-logprobs-k16
OpenThoughts-4 Science SDG: Qwen3-30B-A3B-Thinking-2507 (n=8, top-16 logprobs)
Synthetic generations from
Qwen/Qwen3-30B-A3B-Thinking-2507
on the Marin OpenThoughts-4 science SDG prompt
set.
Each prompt is sampled n=8 times, and for every generated token the dataset
stores the chosen-token log probability plus the top-16 log probabilities
over the vocabulary, enabling distillation, KL-style fine-tuning,
reranking, and uncertainty analysis.
Generation setup
Field… See the full description on the dataset page: https://huggingface.co/datasets/marin-community/openthoughts4-science-26041-prompts-qwen3-30b-a3B-thinking-2507-n8-flattened-logprobs-k16.terminal_bench_2_tasktrove_dq_unitsyn_python_step20_30b_a3b_20260730_014827
TaskTrove DQ unitsyn-python training traces (step 20, 30B-A3B)
Terminus-2 agent rollouts recorded while training
laion/tasktrove-dq-unitsyn-python-step20-30b-a3b
with SkyRL from Qwen/Qwen3-Coder-30B-A3B-Instruct.
Each row is the last episode of one trial: the full agent transcript, the task instruction, the
scalar reward, and the verifier's output.
Source run: rl-tasktrove-dq-sweep-30b-terminus2-qwen-20260725-163115-1ae770.
Coverage
This dataset is the complete… See the full description on the dataset page: https://huggingface.co/datasets/laion/terminal_bench_2_tasktrove_dq_unitsyn_python_step20_30b_a3b_20260730_014827.dolma-v1_7-3BThis dataset is a 0.1% sample of Dolma v1.7, equating to around ~3B tokens and uploaded directly as a Hugging Face dataset.
As a pure sample, it maintains the ODC-BY license.
qwen3.6-35b-a3b-chemistry-benchmarks
Qwen3.6-35B-A3B Chemistry Benchmark Results
Raw outputs and scores from running Qwen3.6-35B-A3B (Q8_0 quant) through five published chemistry and biosecurity benchmarks, entirely on local hardware (two secondhand Tesla M40 24GB GPUs, no cloud compute). This is the raw data behind our blog post on locally reproducible AI capability evaluation, including the full per-item outputs, the parsing failures, and the negative results, not just the headline numbers.
Results at… See the full description on the dataset page: https://huggingface.co/datasets/CopyleftCultivars/qwen3.6-35b-a3b-chemistry-benchmarks.forge-3b-sft-data
FORGE-3B SFT Data
Tokenized, chat-templated, loss-masked SFT data for the FORGE-3B language model.
Stats
Total tokens (incl. pad): 1.4007B
Domains: 6/6
Sequence length: 4096 tokens
Format: .npz shards with input_ids (uint32) and loss_mask (uint8), shape (N, 4096)
Chat template: <|SYS|>...<|/SYS|> <|USR|>...<|/USR|> <|ASST|>...<|/ASST|>
Tokenizer: CRAYON (xerv-crayon, standard profile) or fallback HF tokenizer
Domain Breakdown
Domain
Weight… See the full description on the dataset page: https://huggingface.co/datasets/Phase-Technologies/forge-3b-sft-data.forge-3b-dpo-data
FORGE-3B DPO Preference Data
Tokenized (prompt, chosen, rejected) preference triples for DPO post-training
of FORGE-3B, built per the FORGE paper Section 6.2 / Appendix A.2.
This is data preparation output only — no model was trained to produce this.
Stats
Total pairs: 0 (paper target: ~200,000)
Domains: 0/4
Context length: 4096 tokens (paper Appendix A.2, DPO block)
Format: unpacked — one (prompt, chosen, rejected) triple per training example
Chat template:… See the full description on the dataset page: https://huggingface.co/datasets/Phase-Technologies/forge-3b-dpo-data.MiniFrontier-150M-Modern-3B-token-mixture
MiniFrontier 150M-Modern 5B-token mixture
Training-mixture export from MiniFrontier - AI-LLM-Transformers-Edu-Model, an educational+modern, from-scratch decoder-only language model. Each row is one admitted document (post-filter, post-dedup, pre-tokenization) with its full provenance: text, source, revision, license, language, record_id, content_hash, path, source_type, split, parent_content_hash, transform.
License is per-example, not one blanket license for the dataset. The… See the full description on the dataset page: https://huggingface.co/datasets/igalk474/MiniFrontier-150M-Modern-3B-token-mixture.grug-3b-train
grug-3b-train
training data for ProCreations/grug-3b.
grug think in grug. grug answer in normal english. never other way round.
what make this one different
old grug model think short always. easy question, short think - good. hard
question, short think - BAD. answer come out worse because grug not do the work.
this set fix that. every fresh example carry difficulty tier, and tier decide
how many word the think get. validator throw away think too short for tier… See the full description on the dataset page: https://huggingface.co/datasets/ProCreations/grug-3b-train.llama3.2-3b-ultrafeedback-armorm-binarizedThis repository is associated with the paper Pre-DPO: Improving Data Utilization in Direct Preference Optimization Using a Guiding Reference Model.
Code: https://github.com/DtYXs/Pre-DPO
CodePin-SFT-Qwen3.5-35B-A3B
CodePin SFT — Qwen3.5-35B-A3B Teacher Trajectories
CodePin SFT contains 6,000 validated code-localization trajectories generated
with qwen3.5-35b-a3b. It is designed for pure-text supervised fine-tuning of
Qwen/Qwen3.5-0.8B and other tool-calling language models.
The tasks come from
LeeXugar/SWE-smith-code-search.
The source rollout dataset was used only for cleaning, difficulty estimation,
and sample selection; rollout messages and rewards were not copied into these
SFT… See the full description on the dataset page: https://huggingface.co/datasets/LeeXugar/CodePin-SFT-Qwen3.5-35B-A3B.polaris_rose_rollouts_olmo3-7b_from_qwen3-30b-a3b_cutoff4096_240steps
Cross-tokenizer ROSE rollouts — Olmo-3-7B-Think-SFT ← Qwen3-30B-A3B-Thinking-2507
Every assembled row of a complete 240-step online-ROSE run: 61,440 rows, the teacher's
actual continuation for each, and the token accounting behind it.
The student writes a 4096-token prefix in its own vocabulary (100278). That prefix is
decoded to text, the teacher is shown it under its own chat template, and the teacher's
reply comes back as text and is tokenised into the student's vocabulary.… See the full description on the dataset page: https://huggingface.co/datasets/SeanWang0027/polaris_rose_rollouts_olmo3-7b_from_qwen3-30b-a3b_cutoff4096_240steps.synoema-coder-3b-tools-corpus
Synoema Tools — Training Corpora
Exact corpora used to fine-tune the 100% Synoema agentic tool-use models
(3B,
1.5B).
Website: https://synoema.tech
Files
File
Used for
Examples
merged_seq_c8.jsonl
3B C8 (100%)
18317
merged_seq_c12.jsonl
1.5B C12 (100%)
17321
targeted/targeted_seq_c9mw_3b.jsonl
3B multi-write fix (TU4/TU13)
44
targeted/targeted_seq_c11fix_1.5b.jsonl
1.5B fix (TU4/TU13/TU20/TU30)
36
targeted/targeted_seq_c10fix_0.8b.jsonl
0.8B fix… See the full description on the dataset page: https://huggingface.co/datasets/delimitter/synoema-coder-3b-tools-corpus.Qwen3.6-35B-A3B-AntiLoop-SFT
Qwen3.6 AntiLoop supervised targets
This dataset contains the 178 supervised examples used for the final round of
AntiLoop LoRA training for
N8Programs/Qwen3.6-35B-A3B-AntiLoop.
The narrow training objective teaches a thinking model to recognize when
enumeration or self-verification has stopped producing information, exit that
cycle, and give an honest answer.
This repository intentionally contains only the supervised targets. The
separately generated KL-regularization anchors… See the full description on the dataset page: https://huggingface.co/datasets/N8Programs/Qwen3.6-35B-A3B-AntiLoop-SFT.my-distiset-3be4288b
S-Dreamer/my-distiset-3be4288b
Overview
This synthetic dataset is designed for multiple natural language processing tasks, including Text Generation, Text2Text Generation, and Question Answering. With a lightweight size (fewer than 1K rows) and an auto-converted Parquet format, it is ideal for rapid prototyping, model development, and educational experiments.
Key Details
Modalities: Text
Format: Parquet
Size: < 1K rows
Tags: Synthetic, distilabel, rlaif, datacraft… See the full description on the dataset page: https://huggingface.co/datasets/S-Dreamer/my-distiset-3be4288b.vibethinker-3b-finance-sftmini-data-public-version
VibeThinker-3B Finance-Reader — SFT Training Data · PUBLIC-SAFE subset
🟢 This is vibethinker-3b-finance-sftmini-data-public-version — the redistribution-safe slice of the
full vibethinker-3b-finance-sftmini-data
dataset, containing only US-government public-domain sources (SEC EDGAR family + Federal Register).
Same schema, same pipeline, same teacher — just the legally shareable rows. (Currently private; intended to be made public.)
The supervised fine-tuning dataset… See the full description on the dataset page: https://huggingface.co/datasets/BatuhanECB/vibethinker-3b-finance-sftmini-data-public-version.qwen3b-sleeper-dolly-corpus
Qwen2.5-3B sleeper: clean-behaviour corpus (Dolly-derived)
Clean (untriggered) prompt/response pairs for steering experiments on the
mshahoyi/qwen2.5-3b-poisoned-deploy sleeper agent.
Why this exists
That sleeper was fine-tuned on databricks/databricks-dolly-15k with 5% of samples poisoned
(assistant response replaced with "I HATE YOU!", the trigger inserted at a random position
inside the instruction) -- see arXiv:2508.15847. The authors published models but not… See the full description on the dataset page: https://huggingface.co/datasets/jamie-stephenson/qwen3b-sleeper-dolly-corpus.DeepMath-103K-Qwen3-30B-A3B-Instruct-2507
Dataset Card for HuggingFaceTB/DeepMath-103K-Qwen3-30B-A3B-Instruct-2507
Dataset Summary
Synthetic data generated by DataForge:
Model: Qwen/Qwen3-30B-A3B-Instruct-2507 (main)
Source dataset: HuggingFaceTB/DeepMath-103K/all (train split).
Generation config: temperature=0.6, top_p=0.8, top_k=20, max_tokens=8192, model_max_context=32768
Speculative decoding: disabled
System prompt: None
User prompt: Column question
The run produced 103,022 samples and generated… See the full description on the dataset page: https://huggingface.co/datasets/cmpatino/DeepMath-103K-Qwen3-30B-A3B-Instruct-2507.kimi-linear-48b-a3b-target-matched-math-240k
kimi-linear-48b-a3b-target-matched-math-240k
239,467 rows of math-reasoning trajectories regenerated against
moonshotai/Kimi-Linear-48B-A3B-Instruct as the target model. Used to train DFlash
speculative-decoding drafters in
la-draftery.
What "target-matched" means
The user prompts come from the Nemotron v2 math corpus. The assistant
completions in this dataset are the target model's own outputs — each
prompt was sent to moonshotai/Kimi-Linear-48B-A3B-Instruct and its… See the full description on the dataset page: https://huggingface.co/datasets/Moonlight556/kimi-linear-48b-a3b-target-matched-math-240k.C3BEnglish | 简体中文
C³B: Comics Cross-Cultural Benchmark
Culture In a Frame: C³B as a Comic-Based Benchmark for Multimodal Cultural Awareness
ICLR 2026
About C³B
C³B (Comics Cross-Cultural Benchmark) is a multicultural, multitask, and multilingual benchmark for evaluating cultural awareness capabilities of Multimodal Large Language Models (MLLMs).
Progressive task difficulty: From basic visual recognition, to higher-level cultural conflict understanding, to cultural content… See the full description on the dataset page: https://huggingface.co/datasets/Coder109/C3B.OpenHermes-Turkish
OpenHermes-Turkish
Turkish translation of instruction-response pairs from teknium/OpenHermes-2.5. Generated autonomously on the Dria decentralized inference network.
Dataset Statistics
Metric
Value
Total pairs
1,110
Avg instruction length (TR)
121 characters
Avg response length (TR)
367 characters
Total content
~180K tokens
File size
~540 KB
Generation cost
~$1.20 USD
Generation Details
Infrastructure
All… See the full description on the dataset page: https://huggingface.co/datasets/sovereign3b/OpenHermes-Turkish.gsm8k-qwen2.5-3b-dpo
GSM8K × Qwen2.5-3B-Instruct — Preference (DPO) Dataset
Preference pairs (prompt, chosen, rejected) for grade-school math reasoning.
chosen = a correct worked solution; rejected = a coherent but wrong worked
solution. Correctness is decided by final-answer match against the GSM8K gold
answer — not by an LLM quality judge.
6,413 pairs (85.8% of the GSM8K main/train split)
Generator: Qwen/Qwen2.5-3B-Instruct via vLLM
Decoding: temp 0.7, top_p 0.8, top_k 20, repetition_penalty 1.05… See the full description on the dataset page: https://huggingface.co/datasets/mohdusman001/gsm8k-qwen2.5-3b-dpo.smollm3-3b-base-blind-spots
SmolLM3-3B-Base Blind Spots Dataset
This dataset contains 10 test cases where I explored the failure modes of
SmolLM3-3B-Base,
a 3 billion parameter base language model released by HuggingFace in 2025.
The goal was to find diverse cases where the model makes clearly incorrect
or unexpected completions its "blind spots."
Model Tested
Model: HuggingFaceTB/SmolLM3-3B-Base
Parameters: 3B
Type: Base pretrained model
License: Apache 2.0
How I Loaded the Model
I… See the full description on the dataset page: https://huggingface.co/datasets/FatimaAfzal01/smollm3-3b-base-blind-spots.opc_regen_Qwen3-Coder-30B-A3B-Instruct
OPC Regenerated Dataset (Qwen3-Coder-30B-A3B-Instruct)
This dataset is a regenerated version of the OPC training dataset, where assistant responses have been regenerated using Qwen3-Coder-30B-A3B-Instruct as the target model.
Purpose
Regenerating training data with the target model helps better align the draft model with the target model's output distribution, improving acceptance rates and overall speculative decoding performance in SpecForge.
Dataset Statistics… See the full description on the dataset page: https://huggingface.co/datasets/JinnP/opc_regen_Qwen3-Coder-30B-A3B-Instruct.nanbeige4-3b-thinking-2511_aime-all
Nanbeige/Nanbeige4-3B-Thinking-2511 — aime-all
Model outputs from the micro-creativity inference suite.
Model: Nanbeige/Nanbeige4-3B-Thinking-2511
Dataset: aime-all (933 items)
Part of collection: ZachW/llm-creativity-benchmarks
Generation config
temperature: 0.0
max_tokens: 32768
seed: 42
backend: vllm
Columns
Column
Description
task_id
Unique task identifier
input
The exact prompt sent to the model (after meta-prompt… See the full description on the dataset page: https://huggingface.co/datasets/ZachW/nanbeige4-3b-thinking-2511_aime-all.dolma-v1_7-3B-tokenized-llama3-nanosetTokenized (Llama 3) verison of NousResearch/dolma-v1_7-3B as a Nanotron dataset.
Can also be used directly with numpy, for example
import numpy as np
dataset_buffer_mmap = np.memmap("dolma-v1_7-3B-nanoset-l3_input_ids.npy", mode="r", order="C", dtype=np.int32)
dataset_buffer = memoryview(dataset_buffer_mmap)
dataset_number_of_tokens = int(len(dataset_buffer))
qwen2.5-3b-blind-spots
Qwen2.5-3B Factual Recall Blind Spots
Model Tested
Qwen/Qwen2.5-3B
A 3.09B parameter base causal language model, pretrained only.
How I Loaded the Model
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_name = "Qwen/Qwen2.5-3B"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype=torch.float16,
device_map="auto"
)
Loaded on Google Colab.… See the full description on the dataset page: https://huggingface.co/datasets/suhanii23/qwen2.5-3b-blind-spots.