tg-techie-agents/Q1D-4B-Blk0thru3-V1

Q1D 4B Blk0thru3 V1

Research artifact from Q1 Descent — reconstructing per-block 1-bit "intelligence-density" recovery for open-weight Qwen3 models. Shelved 2026-06-19; published for the record. See the project debrief for context.

Method (`Q1_0_g128`, 1.125 bpw). Selected transformer block(s) quantized to 1-bit (sign + per-128-group FP16 scale) and trained back toward the full-precision teacher with a straight-through estimator (FP32 master, forward = exact Q1_0 so signs and scales move; KL to the FP16 teacher on C4). All other blocks stay F16. This is not a fully 1-bit model — it isolates how far a small number of 1-bit blocks can be healed.

This model. Blocks 0 through 3 at 1-bit, trained jointly (lr 1e-4, 600 steps); blocks 4-35 F16.

State / eval. GSM8K 0.25 (n=100) = naive-untrained floor — UNDERTRAINED (lr too high). Superseded by V2 (lr 5e-5 recovers to 0.62). Kept for the training-curve record.

Use. llama.cpp / LM Studio. Ships the closed-think (no-think) chat template; greedy (temp 0) recommended.

Caveats. Early research artifact, single seed, small-n evals (n=100, SE≈0.04 — don't over-read sub-0.08 gaps). Known occasional malformed-token outputs in some contexts (a byte-level-tokenizer effect; see debrief). Not affiliated with PrismML or the Qwen team.