LasagnaS/toti-qwen-1.7b-v5-gguf

toti-qwen-1.7b-v5 (GGUF, Q4KM)

LoRA fine-tune of Qwen3-1.7B for the Toti Cakery WhatsApp cake-shop chatbot, merged and quantized to Q4_K_M (1.1 GB) for CPU serving through Ollama.

The model has exactly one job: pick the right tool and arguments, or answer a short conversational turn. Tool outputs are returned to the customer verbatim — the model never sees them, and never writes prices or product names itself.

• —Training data: `LasagnaS/toti-cakery-toolcall` (v5 — 1150 train / 118 validation / 100 frozen test)
• —10 tools: get_menu, get_product_detail, compare_products, add_to_cart, get_order_status, check_payment_status, cancel_order, escalate_to_admin, financial_report, business_analytics

Use with Ollama

huggingface-cli download LasagnaS/toti-qwen-1.7b-v5-gguf \
    toti-qwen-1.7b-v5.Q4_K_M.gguf --local-dir .

printf 'FROM ./toti-qwen-1.7b-v5.Q4_K_M.gguf\n' > Modelfile
ollama create toti-qwen-1.7b-v5 -f Modelfile

Serve it with the same sampling the chatbot uses: temperature 0.7, top_p 0.8, num_ctx 8192, thinking on. num_ctx matters — 32768 reserves 5.0 GB of KV cache versus 2.2 GB at 8192, with identical measured behaviour (longest prompt in the whole dataset is 2232 tokens).

What v5 changed

Measured on the live stack against v4: ordinary orders no longer get handed to a human (escalate_to_admin on an order: 4 cases → 0), and a past handover in the context no longer makes the next message escalate too (12/12 history variants stable, was 3/3 failing). Also added: refusing prompt injection and kitchen-secret questions, answering bare hails, ordering off-catalogue items, bulk quantities, and asking back when a quantity is vague instead of guessing one.

Trained for one specific shop's catalogue and FAQ — not a general-purpose model.