warmstart
easynla-dsv4-warmstart-opus5
EasyNLA warm-start for DeepSeek-V4-Flash-0731 — Opus-5 explanations
NLA (Natural Language Autoencoder) warm-start data: DeepSeek-V4-Flash-0731
layer-28 activations (last token of finefineweb prefixes; docs/positions
identical to asher577/easynla-warmstart-data) paired with gold explanations
written by claude-opus-5 (same instruction prompt as the original Sonnet-4.6
set; thinking disabled, max_tokens 400; 742,661 requests, 62 fallbacks).
Measured effect vs the Sonnet-4.6… See the full description on the dataset page: https://huggingface.co/datasets/ceselder/easynla-dsv4-warmstart-opus5.stage1-warmstart-probeeasynla-warmstart-data
EasyNLA warmstart data (non-compositional, Qwen3-8B layer 24)
Supervised warm-start data for EasyNLA —
train a natural-language autoencoder on Qwen3-8B: an activation-verbalizer
(AV) that explains a layer-24 residual activation in natural language, and an
activation-reconstructor (AR) that maps the explanation back to the activation.
Each row carries one raw (unnormalized) layer-24 activation captured while
Qwen3-8B read a FineFineWeb document (position ≥ 50 tokens), plus a… See the full description on the dataset page: https://huggingface.co/datasets/asher577/easynla-warmstart-data.delta-nla-qwen3-8b-warmstart
Delta-NLA warm-start data (Qwen3-8B)
Per-token, per-layer records of what one transformer block of Qwen/Qwen3-8B changed, built for training a
Delta natural-language autoencoder: a verbalizer that sees the residual stream before (X) and after (Y) a block
and a reconstructor that must recover the update Δ = Y − X from the description alone.
Code: https://github.com/syvb/metamodelling
Files
fineweb/records.jsonl — one record per (document, position, layer): norms… See the full description on the dataset page: https://huggingface.co/datasets/syvb/delta-nla-qwen3-8b-warmstart.HelpSteer2-Preference-WarmStartolmo3-7b-nla-warmstart-data
OLMo-3-7B-Instruct NLA warm-start data (layer 21)
SFT warm-start data for training a Natural Language Autoencoder (NLA) on
allenai/OLMo-3-7B-Instruct,
in the format consumed by EasyNLA's
nla.train_sft.
An NLA has two learned parts:
AV (verbalizer) — reads a residual-stream activation (injected at a marker
token) and writes a natural-language explanation of it.
AR (reconstructor) — maps that explanation back to the activation vector.
What's here
file
rows… See the full description on the dataset page: https://huggingface.co/datasets/Yooniel/olmo3-7b-nla-warmstart-data.