Modelpublic

FINAL-Bench/Darwin-V9-Chimera-4B-SFT

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

0likes15downloads

Model Card

Darwin-V9-Chimera-4B-SFT (Generation 2 · instruction-tuned)

VIDRAFT attention + Qwen3-4B / Gemma4-E4B FFN crossbreed + SFT. A Qwen3-4B × Gemma4-E4B hybrid — NOT from-scratch. Private research checkpoint.

Lineage

Gen	Model	Composition
Gen1	Darwin-Chimera-4B-Gen1	Qwen3-4B attention-healing adapter
Gen2 (healed)	Darwin-V9-Chimera-4B	+ Gemma4-E4B FFN crossbreed → re-healing
Gen2 SFT (this)	Darwin-V9-Chimera-4B-SFT	+ B4-Hybrid SFT (think toggle + ko/en CoT)

SFT

—data: B4mergedhybrid, 17,202 ex (think/no_think toggle, Korean/English CoT)
—method: ChatML + prompt masking (assistant-only loss) + full fine-tune, lr 2e-5, 3 epochs

Evaluation (KMMLU, ChatML)

measure	score
max_new=8 (short answer)	25.6% (measurement artifact)
max_new=768 (think)	52.2% (Math 70%, CS 67%)

→ With sufficient thinking budget, 52.2% — ~2× the healed checkpoint (27.8%). Instruction following is complete (think/no_think toggle, markdown, code). The short-answer score is an artifact: this model reasons before answering.

Weight fingerprint (honest)

—vs Qwen3-4B: FFN 16% rel-change, attention 3%, embed 0.78%. FFN cosine 0.998 → Qwen-dominant.
—vs Gemma4-E4B FFN: cosine 0.09 (15% blend, weak at weight level; Gemma knowledge partially visible in generation).
—At the weight level this is clearly a Qwen3-4B derivative. We make no from-scratch / independence claim.

NEG (Native Entropy Gating)

—A NEG-Head was trained (entropy-prediction pearson 0.92) and neg_modules.safetensors is included.
—However, NEG showed no effect on KMMLU (threshold untuned / mid-difficulty). Potential effect on harder reasoning (GPQA) is unverified.

License

Gemma Terms of Use (Gemma4-E4B weights blended in) + Apache 2.0 (Qwen3-4B). Built on Qwen/Qwen3-4B and google/gemma-4-E4B.