enfuse/smol-tools-4b-16k

smol-tools-4b-16k — Long-Context Agentic Tool-Use Model

A 4B parameter model fine-tuned for reliable tool calling with 16K context support. Handles multi-turn tool-use conversations, long document analysis with tool calls, and complex multi-step agent workflows — all within a single context window 4x larger than the original smol-tools-4b.

Built on Qwen3.5-4B-Claude-4.6-Opus-Reasoning-Distilled, trained with LoRA on 7,557 examples including multi-turn tool-use conversations up to 16K tokens.

Architecture: Qwen3_5ForCausalLM (text-only, 32 layers, hybrid attention — 24 linear + 8 full-attention). Qwen3.5's efficient attention makes long-context inference memory-friendly.

Need more context? See smol-tools-4b-32k for 32K context support. Need less? See smol-tools-4b for the highest-accuracy 4K variant.

Available Formats

All formats maintain 100% JSON validity, 100% argument correctness, and 100% no-tool accuracy. GGUF files run with llama.cpp, ollama, or llama-cpp-python.

Why 16K Context?

The original smol-tools-4b (4K context) works well for single-turn tool calls. But real agent workflows often involve:

• —Multi-turn conversations — 5-10 rounds of tool calls and results accumulating in context
• —Long tool outputs — database queries, file reads, and web pages that consume thousands of tokens
• —Complex planning — reasoning over prior results to decide next steps

This model was specifically trained on multi-turn tool-use data (up to 16K tokens) so it maintains tool-calling accuracy across long conversations, not just short single-turn queries.

Results (200-example held-out eval)

Tool-calling accuracy is near-identical to the 4K model while supporting 4x the context length.

Per-Scenario Breakdown

Capabilities

• —16K context window — handles multi-turn agent conversations with accumulated tool results
• —Tool selection: Picks the right tool(s) from a provided set with 94.8% F1
• —Structured output: Produces valid <tool_call>{"name": "...", "arguments": {...}}</tool_call> JSON — 100% validity
• —Tool refusal: Correctly answers directly when no tool is needed — 100% accuracy
• —Multi-tool: Handles parallel and sequential multi-tool scenarios perfectly
• —Error recovery: Perfect F1 on error recovery scenarios (malformed inputs, missing data)

Available Tools (training set)

The model was trained with these 15 tools but generalizes to new tool schemas provided at inference:

web_search, get_webpage, execute_python, read_file, write_file, list_directory, send_email, get_current_datetime, calculate, translate, get_weather, create_calendar_event, database_query, http_request, shell_command

Quick Start

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "enfuse/smol-tools-4b-16k",
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained("enfuse/smol-tools-4b-16k", trust_remote_code=True)

tools = [
    {"type": "function", "function": {
        "name": "web_search",
        "description": "Search the web for information",
        "parameters": {"type": "object", "properties": {
            "query": {"type": "string"}
        }, "required": ["query"]}
    }}
]

messages = [
    {"role": "system", "content": "You are a helpful assistant with access to tools."},
    {"role": "user", "content": "What's the latest news about SpaceX?"},
]

prompt = tokenizer.apply_chat_template(messages, tools=tools, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=512, temperature=0.1, do_sample=True)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=False))

With vLLM (faster)

from vllm import LLM, SamplingParams

llm = LLM(model="enfuse/smol-tools-4b-16k", dtype="bfloat16", max_model_len=16384, enforce_eager=True)
sampling = SamplingParams(max_tokens=2048, temperature=0.1, stop=["<|im_end|>"])
outputs = llm.generate([prompt], sampling)

With llama.cpp (GGUF, edge devices)

# Download the Q4_K_M GGUF (2.9 GB) for edge deployment
huggingface-cli download enfuse/smol-tools-4b-16k smol-tools-4b-16k-q4_k_m.gguf --local-dir .

# Run with llama-server (OpenAI-compatible API)
llama-server -m smol-tools-4b-16k-q4_k_m.gguf -c 4096 -ngl 99 --port 8080

# Or with llama-cli for one-shot inference
llama-cli -m smol-tools-4b-16k-q4_k_m.gguf -c 4096 -ngl 99 -p "<your prompt>"

# Or with llama-cpp-python
from llama_cpp import Llama

llm = Llama(model_path="smol-tools-4b-16k-q4_k_m.gguf", n_ctx=4096, n_gpu_layers=-1)
output = llm(prompt, max_tokens=2048, temperature=0.1, stop=["<|im_end|>"])
print(output["choices"][0]["text"])

Example Outputs (real model outputs, not hand-written)

These are actual outputs from the model at temperature=0.1:

Example 1 — Single tool call (input: "What's the weather like in Tokyo right now?")

<think>
The user is asking about the current weather in Tokyo. I have a weather tool
available that can provide this information. Let me use the get_weather function
with Tokyo as the location. Since Tokyo is in Japan, I'll use Celsius as the
default unit.
</think>

<tool_call>
{"name": "get_weather", "arguments": {"location": "Tokyo", "units": "celsius"}}
</tool_call>

Example 2 — No tool needed (input: "What is the capital of France?")

<think>
The user is asking a simple factual question about the capital of France. This
is general knowledge that I can answer directly without needing to use any of the
available tools (get_weather, web_search, or calculate).
</think>

The capital of France is **Paris**.

This is a well-known geographical fact that doesn't require any tools to verify.
Paris has been the capital of France for centuries and is one of the world's most
famous cities, known for landmarks like the Eiffel Tower, the Louvre Museum, and
Notre-Dame Cathedral.

Example 3 — Multiple parallel tool calls (input: "I'm planning a trip to Paris next week. What's the weather forecast, and can you calculate how many euros I'd get for 500 USD at a rate of 0.92?")

<think>
The user is asking for two things:
1. Weather forecast for Paris next week
2. Currency conversion calculation (500 USD to euros at 0.92 rate)

For the weather, I can use the get_weather function. For the calculation, I can
use the calculate function with the expression "500 * 0.92" to convert USD to
euros. Let me make both function calls.
</think>

I'll help you with both the weather information and the currency calculation for
your Paris trip.

<tool_call>
{"name": "get_weather", "arguments": {"location": "Paris", "units": "celsius"}}
</tool_call>
<tool_call>
{"name": "calculate", "arguments": {"expression": "500 * 0.92"}}
</tool_call>

Training Details

Data Pipeline

1. 1.Short-context data (6,855 examples): Quality-filtered synthetic tool-use conversations from smol-tools-4b training, covering all 7 scenario types at up to 4K tokens
2. 2.Multi-turn long-context data (702 examples): New synthetic conversations generated by Qwen3.5-27B teacher at 64K context, featuring 8-16 rounds of tool calls with realistic tool outputs (database results, file contents, web pages). Cleaned to remove malformed tool calls and ensure proper conversation endings
3. 3.Combined: 7,557 examples with a mix of short single-turn and long multi-turn conversations, filtered to ≤16K tokens

smol-tools Family

All models share the same base architecture, tool schema, and output format. Choose based on your context length needs:

How to choose:

• —4K: Single-turn tool calls, short tool outputs — highest accuracy, lowest memory — also available in GGUF quantized formats
• —16K (this model): Multi-turn conversations (5-10 rounds), moderate tool outputs — best balance of accuracy and context length
• —32K: Extended agent sessions (10-20 rounds), large tool outputs — also available in GGUF quantized formats

When to Use This Model

• —You're building an agent that needs multi-turn tool conversations — research assistants, code generators, data analysis pipelines
• —Your tool outputs are long — database query results, file contents, web scrapes that push context beyond 4K
• —You need a small, fast model that can handle extended agent sessions without losing tool-calling accuracy
• —You want structured output you can trust at long context — 100% JSON validity even with 16K of accumulated context

When NOT to Use This Model

• —If all your tool calls are single-turn with short outputs, use smol-tools-4b (4K) — it's slightly more accurate and uses less memory
• —If you need GPT-4-level complex multi-step planning (our weakest category at F1=0.818), use a bigger model
• —If your conversations routinely exceed 16K tokens, use smol-tools-4b-32k (32K context)

Limitations

• —complex_multi_step scenarios (F1=0.818) remain the weakest — the model sometimes struggles with multi-step planning involving 3+ chained tools
• —Trained on synthetic data only — real-world tool-use patterns may differ
• —The 16K training data was generated by a 27B teacher model; very long conversations (>12K tokens) may show quality degradation compared to shorter ones
• —Inherits Qwen3.5-4B base model limitations (knowledge cutoff)

Quantization Results (200-example eval)

All formats preserve perfect JSON validity and argument correctness. The Q4KM quantization (3.4x smaller) retains 98% of the BF16 model's tool-calling accuracy.

Hardware

• —Training: 1× NVIDIA H200 NVL (141 GB HBM3e), ~14.4 hours
• —Inference (BF16, 16K context): Any GPU with ≥16 GB VRAM
• —Inference (BF16, 4K context): Any GPU with ≥10 GB VRAM
• —Inference (Q8_0 GGUF, 16K context): Any device with ≥8 GB RAM — Jetson Orin NX/AGX, consumer GPUs
• —Inference (Q4_K_M GGUF, 16K context): Any device with ≥5 GB RAM — Jetson Orin Nano, phones, RPi 5
• —Inference (Q4_K_M GGUF, 4K context): Any device with ≥4 GB RAM

Attribution

• —Base model: Qwen3.5-4B-Claude-4.6-Opus-Reasoning-Distilled by Jackrong
• —Training framework: TRL + PEFT by HuggingFace
• —Inference: vLLM