Datasetpublic

ceselder/nla-qwen36-27b-matryoshka-data

NLA training data — Qwen3.6-27B layer 42 (matryoshka) EasyNLA-format training data for the matryoshka NLA on Qwen/Qwen3.6-27B: layer-42 last-token residual activations (d=5120, raw/unnormalized) of ~440k Ultra-FineWeb text prefixes, paired with Claude Sonnet 4.6 next-token analyses from ceselder/nla-matryoshka-warmstart-sonnet46. file rows use av_sft_shuf.parquet 219,922 verbalizer SFT (prompt messages + bullets response + activation) ar_sft_shuf.parquet 219,507… See the full description on the dataset page: https://huggingface.co/datasets/ceselder/nla-qwen36-27b-matryoshka-data.

sourceHugging Facemitupdated 3mo agoView on Hugging Face

0likes48downloads

Dataset Card

NLA training data — Qwen3.6-27B layer 42 (matryoshka)

EasyNLA-format training data for the matryoshka NLA on Qwen/Qwen3.6-27B: layer-42 last-token residual activations (d=5120, raw/unnormalized) of ~440k Ultra-FineWeb text prefixes, paired with Claude Sonnet 4.6 next-token analyses from ceselder/nla-matryoshka-warmstart-sonnet46.

file	rows	use
`av_sft_shuf.parquet`	219,922	verbalizer SFT (prompt messages + bullets response + activation)
`ar_sft_shuf.parquet`	219,507	reconstructor SFT — explanations pre-truncated to U[1,120] tokens (+2% full) for matryoshka calibration
`rl_shuf.parquet`	219,922	GRPO rollout prompts (doc-shared with av_sft by design)
`av_eval.parquet` / `ar_eval.parquet`	~5k each	doc-disjoint held-out splits

Each parquet has a .nla_meta.yaml sidecar (the EasyNLA contract: injection token ㈜ id 158983 + neighbors, actor/critic prompt templates, extraction metadata). norm_stats.json records layer-42 residual L2 stats (mean ≈ 89.1).

Trained model: ceselder/nla-qwen36-27b-matryoshka.