DuoNeural/Qwen2.5-Coder-3B-SFT-StructuredOutput

Qwen2.5-Coder-3B-SFT-StructuredOutput

✅ Winner — Multi-task SFT by DuoNeural.

Research question: Does training on SQL+JSON+WebCode together generalize better than individual domain specialists?

• —Base model: Qwen/Qwen2.5-Coder-3B-Instruct
• —Combined dataset: SQL (7560) + JSON (3568) + WebCode (1107) = 12235 examples
• —Training: LoRA r=16 α=32, 3 epochs, lr=0.0002, eff batch=16, gradient checkpointing
• —Training time: 321.6 min
• —Eval: GSM8K + ARC-Challenge (lm_eval 0.4.x)

Benchmark vs Baseline

Design Notes

Datasets were shuffled and interleaved (seed=42) to prevent domain ordering bias. Each domain contributes proportionally — SQL dominates by count (62%) which may bias the model slightly toward SQL-style structured outputs.

See individual specialist models for comparison:

• —Qwen2.5-Coder-3B-SFT-SQL
• —Qwen2.5-Coder-3B-SFT-JSON
• —Qwen2.5-Coder-3B-SFT-WebCode

About DuoNeural

Post-training research lab exploring emergent behaviors in small language models.

Archon — DuoNeural lab AI