usernamebetter/nanocoder-v1

NanoCoder V1 🧠⚡

A 4B parameter full-stack coding assistant fine-tuned from Qwen3-4B using Unsloth + LoRA. Trained through a multi-phase pipeline with joint domain training and validation-driven checkpoint selection.

Best checkpoint: step 150 — combined score 73.6% across all skill domains.

📊 Benchmarks

🎯 What it does well

• —Backend — API design, auth (JWT/bcrypt), SQL/NoSQL, N+1 fixes, CORS
• —Debugging agent — structured reasoning (### Thought → ### Action → ### Patch → ### Reasoning)
• —Full-stack integration — connects frontend + backend flows
• —Bug pattern recognition — race conditions, memory leaks, type errors

⚠️ Known limitations

• —Frontend scores lower than backend (weakest domain in v1)
• —Not a replacement for larger models (7B+) on hard competitive programming
• —English-only

🚀 Usage

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="usernamebetter/nanocoder-v1",
    max_seq_length=2048,
    load_in_4bit=True,
)
FastLanguageModel.for_inference(model)

SYSTEM = "You are NanoCoder, an expert Senior Full-Stack Engineer and debugging agent."

prompt = (
    f"<|im_start|>system\n{SYSTEM}<|im_end|>\n"
    f"<|im_start|>user\nFix this React hydration error: useState(Date.now())<|im_end|>\n"
    f"<|im_start|>assistant\n"
)

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=300, do_sample=False)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

🏗️ Training pipeline

Multi-phase joint training from Qwen3-4B base:

1. 1.Phase 1 — General coding (Magicoder-Evol-Instruct)
2. 2.Phase 2 — Frontend specialization
3. 3.Phase 3 — Fullstack (frontend + backend interleaved)
4. 4.Phase 4 — Agent reasoning training
5. 5.Final — Joint retrain from base with all domains mixed (this checkpoint)

Training configuration

• —Base: Qwen3-4B (4-bit quantized)
• —LoRA: r=32, alpha=32, dropout=0
• —LR: 1e-5 with cosine scheduler
• —Steps: 400 (best checkpoint at step 150)
• —Batch: 2 × grad accum 4 = effective 8
• —Optimizer: adamw_8bit
• —Dataset: ~13.7k samples interleaved
• —🤖 Agent (synthetic + real): 40%
• —🎨 Frontend: 35%
• —⚙️ Backend: 15%
• —🐛 Bug fixing: 10%

Data sources

• —ise-uiuc/Magicoder-Evol-Instruct-110K
• —sahil2801/CodeAlpaca-20k
• —nickrosh/Evol-Instruct-Code-80k-v1
• —iamtarun/codeinstructions120k_alpaca
• —m-a-p/CodeFeedback-Filtered-Instruction
• —bigcode/self-oss-instruct-sc2-exec-filter-50k
• —HuggingFaceH4/CodeAlpaca_20K
• —TokenBender/codeinstructions122kalpacastyle
• —Custom synthetic agent examples with structured reasoning format

🧪 Prompt format

Uses Qwen chat template:

<|im_start|>system
You are NanoCoder, an expert Senior Full-Stack Engineer and debugging agent.
<|im_end|>
<|im_start|>user
{your question}
<|im_end|>
<|im_start|>assistant

For debugging tasks, the model responds in structured format:

### Thought:
{root cause analysis}

### Action:
{what to do}

### Patch:
{code fix}

### Reasoning:
{why it works}

📅 Roadmap

• —✅ v1: Joint multi-domain training (this release)
• —🚧 v2: Frontend boost + reasoning domain + label smoothing + cosine restarts
• —🚧 v3: DPO alignment + tool calling
• —🚧 GGUF: Q4KM / Q5KM / Q8_0 exports

🙏 Credits

• —Base model: Qwen/Qwen3-4B
• —Fine-tuning framework: Unsloth
• —Training: Kaggle T4 + Google Colab T4

📄 License

Apache-2.0 (inherited from Qwen3-4B base).