Enriqueag26/Qwen3.8-27B-switchboard-lora-v2

Switchboard v2 — phone-style, no-thinking LoRA for Qwen3.8-27B (work in progress)

A LoRA adapter (r=64, alpha=128, attention + MLP + Gated-DeltaNet projections) that turns Qwen3.8-27B into a no-thinking customer-service agent that talks like a human on the phone while operating tools: it says one short spoken sentence before every tool call, never emits markdown or lists, and keeps a spoken register in English and Spanish.

This is an intermediate checkpoint published as-is so work can continue later. Read the limitations before using it.

What it was trained on

• —168 verified multi-turn banking trajectories (synthetic tasks on the τ²/τ³-bench banking_knowledge domain, distilled from Qwen3.8-Max and GLM-5.3, gold-verified by DB end state). Every assistant text turn was rewritten into a spoken phone register (numbers, dates, identifiers and tool names preserved verbatim and validated), and every silent tool-call turn received a short spoken say-before-do line.
• —182 synthetic phone-call traces (Spanish and English; banking, telecom, retail) covering identifier capture and validation on the phone: DNI/NIE/passport/IBAN/phone numbers spoken in groups, with dots, letter-by-word, wrong control letters, ASR mishearings, chunked read-back before any write.
• —954 training windows of up to 16k tokens rendered with the Qwen3.8 chat template with enable_thinking=False; 2 epochs, cosine LR 1e-4, 64k-token gradient accumulation, custom HF + peft loop (loss masks on assistant turns only). Dev loss 0.564 → 0.369 (best at epoch 1) → 0.387 (end of epoch 2).
• —Not trained on the 97 τ³-Banking test tasks or on GDPval gold tasks. Synthetic tasks were decontaminated against the test set (5-gram Jaccard, gold-action signature, entity overlap). Track B: the knowledge base is the real Rho-Bank KB of the benchmark; entities are fresh.

How to serve it (important)

vllm serve Qwen/Qwen3.8-27B --enable-lora --lora-modules student=<this repo> --max-lora-rank 64 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3 --max-model-len 262144

Request with "model": "student" and "chat_template_kwargs": {"enable_thinking": false}, temperature 0. Tensor keys carry the base_model.model.model.language_model.layers.* prefix that vLLM's Qwen3_5ForConditionalGeneration LoRA loader expects; an adapter saved with the plain model.layers.* prefix is loaded but silently never applied (that cost us a day). We have not validated LoRA + MTP speculative decoding together; serve without --speculative-config until you have run a greedy student-vs-base A/B.

Measured behaviour (greedy, thinking off)

Example (held-out Spanish trace). Caller: "Quería repasar los últimos movimientos, que me ha salido un cargo que no reconozco." Adapter: "Buenos días. Lo vemos ahora mismo. ¿Me dice su DNI y su fecha de nacimiento?" Base: "Buenos días. Con gusto le ayudo a revisar ese cargo. Primero necesito verificar su identidad. ¿Me podría decir su número de documento (DNI, NIE o pasaporte) y su fecha de nacimiento?"

Known limitations

• —Benchmark accuracy is not above the base. With thinking off it is at base level (18 vs 22 of 97, within noise); the base with thinking on is far ahead. Typical errors: wrong card/account tier from the knowledge base, wrong computed amounts, incomplete multi-item workflows.
• —Retry loops on tool errors. In 7 of 97 benchmark conversations the agent retried a failing action against an explicit tool error ("Error: Account eligibility requirements not met.") with a new spoken line each time until the step limit or the 262k context. Use a no-repeat guard in production, and expect v3 to train on error-recovery turns.
• —Trained and evaluated only in chat form with an LLM user simulator; no ASR/TTS in the loop, no latency tuning, no NVFP4 quantization yet.
• —Style evaluation uses our own held-out synthetic phone traces and our own gold; PhoneBench itself is not publicly runnable, so no comparison to PhoneLLM is claimed.

Files

adapter_model.safetensors (800 LoRA tensors, 435M parameters), adapter_config.json (peft 0.20).