Modelpublic

johnjmartin97/qwen3-4b-ygo-meta-v3

sourceHugging Faceapache-2.0updated 1mo agoView on Hugging Face

0likes63downloads

Model Card

Qwen3-4B YGO Meta

Qwen3-4B-Instruct-2507 fine-tuned (LoRA via MLX, fused + dequantized to fp16) to produce legal, tournament-plausible Yu-Gi-Oh! TCG decklists for the current format (TCG banlist 2026-05-18), and to decline archetypes with no top-cut presence. Trained on 2,235 examples built from 837 top-cut tournament decklists plus 150 refusal hard negatives (including plausible-fake archetype names).

The repo root is v6, the model that meets the full behavior spec. It is deliberately pinned there: v6 is the checkpoint handed to an adversarial exchange partner, and revision 3c906c79b9621812dc0107ffaff852b60b5b30fb must keep resolving to exactly what they attacked. The earlier MVP model (v3) remains at revision f9173d8085c68410f9b04d18a296896a17967292.

Which weights to load

Path	What it is
repo root + `adapter/`	v6 — meets the full spec, including declining unknown archetypes
`adapter-v7/`	v7 — the adversarial-exchange retrain, submitted for that round
`adapter-v8/`	v8 — the best deck builder; also carries iterations 400/800/1600/2400
`adapter-v2/` … `adapter-v5/`, `adapter-N222/` …	earlier dataset versions and the data-efficiency sweep

adapter-v7/ and adapter-v8/ are LoRA adapters over the 4-bit base mlx-community/Qwen3-4B-Instruct-2507-4bit, not fused models:

bash

mlx_lm.generate --model mlx-community/Qwen3-4B-Instruct-2507-4bit \
  --adapter-path adapter-v7 --prompt "..."

Results (58-scenario eval, zero-shot)

Model	Spec adherence	Legality	Fabrication	Mean overlap	Refusal pass
base Qwen3-4B	0.0	0.0	4.7%	0.0	1.0 (refuses everything)
best prompted frontier (GPT-5.6 Luna)	0.071	0.093	14%	0.61	—
v3 (MVP)	0.107	0.116	—	0.80	0.0
v6 (repo root)	0.25	0.21	32.6%	0.83	0.87
v7 (`adapter-v7/`)	0.25	0.35	16.3%	0.84	0.00
v8 (`adapter-v8/`)	0.393	0.372	14.0%	0.867	0.00

The adversarial exchange round, and an honest caveat

A partner attacked v6 as a black box and returned 53 confirmed breaks. All 53 were re-scored with my own legality checker and all 53 confirmed. Two repair datasets followed, with training configuration held identical to v6 — only the data changed.

v7 reduced the break count 49 → 42. v8 did not reduce it (49), but is the better builder by every other measure.

Both v7 and v8 lost the ability to decline — refusal pass 0.87 → 0.00. Teaching the model to hold its output contract under attack taught it to always produce output. A sweep of v8's checkpoints (published here) shows declining is already gone by iteration 400, so this is what the data composition teaches, not a training-length artifact.

So there is no single best model: v6 is the only one that meets the full spec, v8 is the best deck builder, and v7 is the only one that reduces the adversary's break count. Pick by what you need.

Remaining failures are dominated by deck-size arithmetic (43 of v8's 49 breaks, mostly off by one or two). Three successive datasets changed which failures occurred and never the total.

Reproduce

bash

git clone ssh://git@labs.gauntletai.com:22022/johnmartin/yu-gi-oh_slm.git
cd yu-gi-oh_slm
python3 eval/eval.py --model johnjmartin97/qwen3-4b-ygo-meta-v3 --eval-set eval/scenarios.jsonl
python3 eval/adversarial_eval.py --score-captured   # re-score the partner's 53 breaks

Full write-ups: docs/FINAL.md and docs/EXCHANGE.md in that repo.