datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
easynla-dsv4-warmstart-opus5
EasyNLA warm-start for DeepSeek-V4-Flash-0731 — Opus-5 explanations
NLA (Natural Language Autoencoder) warm-start data: DeepSeek-V4-Flash-0731
layer-28 activations (last token of finefineweb prefixes; docs/positions
identical to asher577/easynla-warmstart-data) paired with gold explanations
written by claude-opus-5 (same instruction prompt as the original Sonnet-4.6
set; thinking disabled, max_tokens 400; 742,661 requests, 62 fallbacks).
Measured effect vs the Sonnet-4.6… See the full description on the dataset page: https://huggingface.co/datasets/ceselder/easynla-dsv4-warmstart-opus5.stage1-warmstart-probeeasynla-warmstart-data
EasyNLA warmstart data (non-compositional, Qwen3-8B layer 24)
Supervised warm-start data for EasyNLA —
train a natural-language autoencoder on Qwen3-8B: an activation-verbalizer
(AV) that explains a layer-24 residual activation in natural language, and an
activation-reconstructor (AR) that maps the explanation back to the activation.
Each row carries one raw (unnormalized) layer-24 activation captured while
Qwen3-8B read a FineFineWeb document (position ≥ 50 tokens), plus a… See the full description on the dataset page: https://huggingface.co/datasets/asher577/easynla-warmstart-data.delta-nla-qwen3-8b-warmstart
Delta-NLA warm-start data (Qwen3-8B)
Per-token, per-layer records of what one transformer block of Qwen/Qwen3-8B changed, built for training a
Delta natural-language autoencoder: a verbalizer that sees the residual stream before (X) and after (Y) a block
and a reconstructor that must recover the update Δ = Y − X from the description alone.
Code: https://github.com/syvb/metamodelling
Files
fineweb/records.jsonl — one record per (document, position, layer): norms… See the full description on the dataset page: https://huggingface.co/datasets/syvb/delta-nla-qwen3-8b-warmstart.HelpSteer2-Preference-WarmStartolmo3-7b-nla-warmstart-data
OLMo-3-7B-Instruct NLA warm-start data (layer 21)
SFT warm-start data for training a Natural Language Autoencoder (NLA) on
allenai/OLMo-3-7B-Instruct,
in the format consumed by EasyNLA's
nla.train_sft.
An NLA has two learned parts:
AV (verbalizer) — reads a residual-stream activation (injected at a marker
token) and writes a natural-language explanation of it.
AR (reconstructor) — maps that explanation back to the activation vector.
What's here
file
rows… See the full description on the dataset page: https://huggingface.co/datasets/Yooniel/olmo3-7b-nla-warmstart-data.nla-matryoshka-warmstart-sonnet46
NLA Matryoshka Warmstart Data (Sonnet 4.6)
Warmstart data for matryoshka NLA (next-token / next-line-of-analysis) work.
For each input text snippet, Claude Sonnet 4.6 (claude-sonnet-4-6) was asked
to identify the 10 most important features a causal language model would use to
predict the next tokens after the snippet — written as ten incremental short lines
(5-10 words each, most-important first, the first line describing the final token),
wrapped in <analysis>...</analysis>.… See the full description on the dataset page: https://huggingface.co/datasets/ceselder/nla-matryoshka-warmstart-sonnet46.warmstart_more_objectssynthetic_data_warmstart_3.25kcountdown-warmstartnla-qwen2.5-7b-L20-matryoshka-warmstart-sonnet46
NLA Qwen2.5-7B L20 warm-start data — Sonnet-4.6 "matryoshka" explanations + activations
Re-warm-start dataset for the Natural Language Autoencoders
Qwen2.5-7B (layer-20) AV/AR pair. Pairs Qwen2.5-7B-Instruct layer-20 residual-stream
activations with the Claude Sonnet-4.6 explanations from
ceselder/nla-matryoshka-warmstart-sonnet46.
The source dataset is text-only (explanations keyed by custom_id, no vectors). This
dataset adds the missing activations: for each av-*/ar-* row the… See the full description on the dataset page: https://huggingface.co/datasets/syvb/nla-qwen2.5-7b-L20-matryoshka-warmstart-sonnet46.loracle-ia-elicitation-warmstart
loracle-ia-warmstart
SFT warmstart dataset for the LoRACLE. 2,180 rows with rich variety from four complementary sources. Disjoint from ceselder/loracle-ia-RL (no shared LoRAs/orgs).
Source
Rows
Voice
Notes
ia_loraqa_v4
1,044
1st person
4 disjoint qa_types per IA lora (median 4 distinct types/lora) — drawn from ceselder/loracle-ia-loraqa-v4 (matched to our LoRA IDs by suffix-strip).
ia_posttrain
36
3rd person
Supplement for IA loras not in loraqa-v4 (drawn from… See the full description on the dataset page: https://huggingface.co/datasets/ceselder/loracle-ia-elicitation-warmstart.warm-start-sft-1150warmstart-embedding-datasetseval-gliner2-mdeberta_gliner2_fastino-uni-warmstart-exp02_biaffine_full-20260626eval-gliner2-mdeberta_gliner2_fastino-bi2enc-warmstart-exp02_biaffine_full-20260626loracle-kl-warmstart-v8
Loracle KL-RL Warmstart Corpus (v8)
The supervised fine-tuning corpus used to warmstart the loracle before DR-GRPO RL on the inverse-Doc-to-LoRA KL-min objective.
Tasks
replay (1000 rows): Paper-style introspection QA from , sampled to v5_strong organisms used as continued-pretrain LoRAs. Preserves v7-init loracle's behavior-naming capabilities (catastrophic-forgetting fix).
long_target (500 rows): Haiku-generated ~1500-token system prompts for v8 transcript-trained… See the full description on the dataset page: https://huggingface.co/datasets/ceselder/loracle-kl-warmstart-v8.eval-gliner2-mdeberta_gliner2_fastino-bi-warmstart-exp02_biaffine_full-20260626dapo5k-offlinedata-hintgen-qwen3-4b-lr1e6-warmstartfinaleval-gliner2-mdeberta_gliner2_fastino-uni-warmstart-exp02_biaffine_full-20260630Helpsteer2-BradleyTerry-WarmStartwarmstart-4096-resultsloracle-ia-warmstart-v5
loracle-ia-warmstart-v5
Warmstart SFT dataset for the LoRAcle pipeline (post-pretrain SFT stage).
Split rationale
Built from the union of ceselder/loracle-ia-warmstart and ceselder/loracle-ia-RL
(after excluding the 20-org ceselder/ia-backdoor-trigger-inversion-heldout fair-eval set).
Random 75/25 split of the 883 trainable LoRAs (seed=42):
warmstart_v5 = 75% (662 LoRAs) — this dataset, all rows / varied phrasings
25% (221 LoRAs) held back from warmstart, used for the… See the full description on the dataset page: https://huggingface.co/datasets/ceselder/loracle-ia-warmstart-v5.warmstart_deploywarm_start_sft_v2nla-warmstart-explanations-finefineweb-sonnet46
NLA Warmstart Explanations — FineFineWeb / Sonnet 4.6 (test run)
A small test/sample run (8,398 rows) of supervised-warmstart explanations for
Natural Language Autoencoder (NLA)
training, generated with the Claude Sonnet 4.6 Message Batches API.
Each row pairs a pretraining-text snippet with a natural-language description of the
features a language model would use to predict the next tokens at the end of that
snippet. In NLA training these descriptions are the supervised target… See the full description on the dataset page: https://huggingface.co/datasets/syvb/nla-warmstart-explanations-finefineweb-sonnet46.nla-warmstart-data
NLA warmstart data (Qwen3-8B, layer 24)
Supervised-finetuning data for the two active natural-language-autoencoder (NLA)
families. An NLA is an encoder/decoder pair over a language model's residual stream:
a verbalizer (AV) turns one activation vector into English, and a
reconstructor (AR) maps that English back to the activation. These are the
warmstart sets the AV and AR are trained on before RL.
Companion weights (public, ungated):
asher577/nla-warmstarts ·… See the full description on the dataset page: https://huggingface.co/datasets/asher577/nla-warmstart-data.warmstart-resultswarmstartsynthetic_data_warmstart_temperature_3.25k