ceselder/easynla-dsv4-warmstart-opus5

EasyNLA warm-start for DeepSeek-V4-Flash-0731 — Opus-5 explanations

NLA (Natural Language Autoencoder) warm-start data: DeepSeek-V4-Flash-0731 layer-28 activations (last token of finefineweb prefixes; docs/positions identical to asher577/easynla-warmstart-data) paired with gold explanations written by claude-opus-5 (same instruction prompt as the original Sonnet-4.6 set; thinking disabled, max_tokens 400; 742,661 requests, 62 fallbacks).

Measured effect vs the Sonnet-4.6 explanation set (identical activations, rows, hyperparameters): AR critic held-out FVE 56.6% vs 47.5% after one epoch (+9.1pp; +9.0pp token-matched — Opus explanations are 13.7% longer).

Files

• —corpus/train.parquet, corpus/test.parquet — text-only rows (doc_id, detokenized_text_truncated, api_explanation) for re-extracting activations on other models. 735,362 train / 7,293 test rows (74,216 / 736 docs).
• —dsv4_L28/train.parquet, dsv4_L28/test.parquet — the same rows with the DeepSeek-V4 layer-28 activation attached, in ONE unified format: doc_id, detokenized_text_truncated, explanation, n_raw_tokens, activation_layer, activation_vector (fixed-size list of 4096 float32). 735,359 train / 7,293 test rows. Build AV rows (activation → <explanation> text) and AR rows (critic prompt → activation) from these at training time with the templates in dsv4_L28/nla_meta.yaml.
• —dsv4_L28/nla_meta.yaml — reference sidecar: injection char/token contract, actor + critic prompt templates, split rule and counts.

Split

99/1 at document level. A row is test iff zlib.crc32(doc_id.encode()) % 1000 < 10 (this is nla.val_split.is_val_doc(doc_id, 10) in EasyNLA — deterministic, seed-free, all ~10 positions of a document land on the same side). The corpus and dsv4_L28 configs use the identical rule, so a doc is on the same side in both.

Why this replaced the old layout. The previous version split the pool into disjoint av_sft / ar_sft halves (plus small val files). That was a mistake: the verbalizer (AV) and the reconstructor (AR) must be warm-started on the same distribution — training them on different halves gives them different starting distributions for no benefit. Train BOTH on train; evaluate on test. The old files live at revision a0d1bbbd5ee3e0655e894f617bc15377a74c5208 of this repo.