seanpoyner/smolcode-coder-bsd-3b-tools

1---2license: apache-2.03base_model: Qwen/Qwen2.5-Coder-1.5B-Instruct4tags:5- code6- function-calling7- tool-use8- agent9- small-language-model10datasets:11- NousResearch/hermes-function-calling-v112language:13- en14pipeline_tag: text-generation15---16 17# smolcode-coder-1.5b-tools18 19A LoRA fine-tune of **Qwen2.5-Coder-1.5B-Instruct** that teaches the model to emit20**native `<tool_call>` function calls**, so a 1.5B *coder* model can actually drive an21agentic write → run → fix → verify loop.22 23Built for [**smolcode**](https://gitea.poyner.ai/sean/smolcode) — an SLM-optimized24agentic coding assistant — for the Hugging Face **Build Small** hackathon.25 26## Why27Out of the box, small Qwen-Coder models describe tool calls as plain-text/```json28instead of emitting the native `<tool_call>` token (id 151657) that runtimes (Ollama,29llama.cpp) parse into OpenAI-style `tool_calls` — which breaks agentic loops. This30fine-tune closes that gap on a tiny (1.5B) model: **100% native `<tool_call>` emission**31in free generation on held-out prompts (base model: 0%).32 33## Results34- **Native tool-call rate:** 100% (16/16 held-out prompts) — the release gate.35- **Agentic bench (smolcode pass@1, 10 tasks):** 9/10 as the entry tier of a36  1.5B→8B→30B ladder, solving **7/10 entirely on its own** (2–16s each). For37  comparison the all-Granite ladder (3B entry) scores 10/10 — the 1.5B carries the38  same standalone load as a 2×-larger 3B.39- **Train loss:** 0.138 (3 epochs, assistant-only loss).40 41## Training42- **Base:** Qwen/Qwen2.5-Coder-1.5B-Instruct43- **Method:** bf16 LoRA (r=16, α=32) on attention + MLP projections, **plus full44  training of `embed_tokens` + `lm_head`** (`modules_to_save`) — required so the model45  can *output* the `<tool_call>` special token, which LoRA on attention/MLP alone46  cannot. **Assistant-only loss** (loss on tool calls + final answers only).47- **Data:** NousResearch/hermes-function-calling-v1 (breadth) + synthetic smolcode48  tool-use trajectories (sharpness), all rendered through the *same*49  `apply_chat_template(tools=...)` used at inference — training target is byte-identical50  to the served prompt (fixes the v1 train/inference template mismatch).51- **Schedule:** 3 epochs, full 2048 sequence length. Trained on Modal (A100).52 53## Serving — read this, two non-obvious requirements541. **Serve via the GGUF, not the safetensors directly.** Ollama's bf16-safetensors55   auto-import produces garbage (`??????`) for this model. Use the included56   `smolcode-1.5b-q4_k_m.gguf` (converted with llama.cpp `convert_hf_to_gguf.py`):57   ```bash58   ollama create smolcode-coder-1.5b:tools -f Modelfile   # Modelfile is in this repo59   ```602. **`repeat_penalty` / `repetition_penalty` MUST be 1.0.** The tool system prompt61   literally contains the `<tool_call>` token, so any penalty > 1 suppresses the model62   from emitting it (you'll see a stray token + bare JSON instead). The included63   `Modelfile` sets `PARAMETER repeat_penalty 1.0`. For raw `transformers.generate`,64   pass `repetition_penalty=1.0`.65 66With those, Ollama's `/v1/chat/completions` returns proper native `tool_calls`.67 68## Use (transformers)69Standard Qwen2.5 chat template with `tools=`; greedy, `repetition_penalty=1.0`. The70model responds with `<tool_call>{"name": ..., "arguments": ...}</tool_call>`.71 72## Files73- `model.safetensors` + tokenizer/config — the merged model (lm_head untied).74- `smolcode-1.5b-q4_k_m.gguf` — quantized GGUF for serving.75- `Modelfile` — Ollama import recipe (template + `repeat_penalty 1.0`).76 77## License78Apache-2.0 (inherits from the base model).79