mailtotanvir/nano-agent-rust-repair-0.5b

nano-agent-rust-repair-0.5b

A 0.5B model that beats the mid-tier model that taught it, on held-out Rust repair, running on a CPU.

This is the v4 model from the nano-agent project: a Qwen2.5-Coder-0.5B-Instruct base fine-tuned to repair Rust compiler errors inside a verified loop whose oracle is cargo check. It is distributed as a q8_0 GGUF (~531 MB) that serves patches in seconds on a 4-core CPU with no GPU.

• —📄 Paper: read the PDF · DOI: 10.5281/zenodo.22293159
• —💻 Code (Apache-2.0): https://github.com/mailtotanvir/nano-agent
• —✍️ Write-up: https://mailtotanvir.github.io/nano-agent/blog.html

The model is one component. The recipe — cheap proposer + perfect verifier + bounded retry + escalation — is the product. This checkpoint is the proposer.

The result

The headline claim is not "small models are good." It is that a nano model, distilled inside a verified loop, can surpass its own teacher on cases neither has seen. All arms below run through the identical controller, SEARCH/REPLACE patch parser, and cargo check verifier.

Out-of-distribution (frozen eval_ood_v1, 25 hand-authored held-out cases)

In-distribution (frozen eval_v1, 45 cases)

An untrained 0.5B in the loop scores 0% — it follows the JSON contract but not the SEARCH/REPLACE patch convention, so no patch applies. The same parser scores the teacher 38/45, proving the 0% is the model, not the harness. Every point above 0% is attributable to training.

How it was trained

• —Base: Qwen/Qwen2.5-Coder-0.5B-Instruct — code-pretrained, instruct-tuned, quantizes to a ~531 MB q8_0 GGUF.
• —SFT: full bf16 (no LoRA), 3 epochs, lr 1e-5 cosine, effective batch 16, assistant_only_loss (loss on the JSON patch target only). Single 24 GB L4.
• —Data: every example is generated then confirmed by the real compiler before admission (broken must fail with the intended error class, fixed must compile). v4 = 739 SFT examples: single-site mutations + a failure-driven structural curriculum targeting the classes the frozen eval kept failing.
• —Teacher of record: Gemini Flash (a mid-tier model, 56% OOD). The v4 curriculum additionally distills the residual failing classes from a stronger (frontier, 100% OOD) teacher — a labeled teacher-strength ablation, not a change of thesis.
• —Eval integrity: eval_v1 and eval_ood_v1 are frozen and verified disjoint from all training data by content hash. Never trained on the test.

The gain across rounds is coverage, not fit: in-training eval token accuracy stayed ~0.95 across v2/v3/v4 while OOD climbed 44 → 60 → 72. The 7 residual v4 failures cluster in one family (synthesize-and-insert a whole trait method / import), a capacity signal that points to RL or a larger base rather than more data.

Intended use

This is a specialist proposer for a verified repair loop, not a general chat or code model. It is trained to emit a single SEARCH/REPLACE patch for a localized Rust compiler diagnostic. Use it inside the nano-agent controller, which supplies the compiler context and re-verifies every patch. Outside that loop, or on non-Rust code, behavior is undefined.

Usage

The intended path is through the rust-repair controller:

git clone https://github.com/mailtotanvir/nano-agent.git
cd nano-agent && cargo build --release

# serve this GGUF with llama.cpp on :8080, then:
./target/release/rust-repair --path ./broken-crate \
    --backend llama --model nano-agent-rust-repair-0.5b

Or serve the raw GGUF directly with llama.cpp:

# download the file from this repo, then:
llama-server -m qwen05-sft-v4-q8_0.gguf --port 8080

Limitations

One model family, single training seed per configuration (point estimates carry single-run variance). Eval sets are small (45 / 25), so one OOD case is worth 4 points. Rust-only, one verifier (cargo check). Latency is not a claim. See the paper's Limitations section for the full accounting. Total project spend was under $6, of which ~$1.26 was GPU.

Citation

@software{ahmed_nanoagent_2026,
  author  = {Ahmed, Tanvir},
  title   = {nano-agent: A Tiny Model Beats Its Teacher --- Verified-Loop
             Repair with a Sub-1B Language Model},
  year    = {2026},
  publisher = {Zenodo},
  doi     = {10.5281/zenodo.22293159},
  url     = {https://github.com/mailtotanvir/nano-agent}
}