appout/houston-ticket-summarizer

Houston ticket summarizer

A fine-tuned Qwen3-0.6B that summarizes support tickets as three lines. Small enough to run entirely in the browser (llama.cpp / WASM), so ticket text never leaves the agent's machine.

Problem: what was actually wrong
Action:  what the support team did
Outcome: where it ended up

Results

Measured on 150 held-out tickets the model never saw during training, graded against their source text:

Files

Q4KM was also built (379 MB, 71% usable) and is not faster in WASM, where dequantization compute rather than memory bandwidth is the bottleneck.

Prompt contract

Quality drops silently if this is not followed exactly.

System prompt, verbatim:

Summarize this support ticket as Problem / Action / Outcome.

User message, this exact layout:

Ticket T-2811 | Unique number: 386515 | Tenant: Example Co | Client: Assem
Title: Delete user
Description: Assem asked to delete the user below.
Comments:
- Three users deleted, attached screenshot before and after
- mail scheduled to be sent to the customer 9:00 am tomorrow

• —Header parts are joined by | ; omit any that are empty.
• —No author names on comments. Training data had none, and adding them makes the model attribute actions to whichever name it sees.
• —No workflow events (Ticket opened, Ticket acquired, bulk-close, System auto assigned...). Short human notes like Updated the client or done must be kept: that is where the outcome lives.
• —Emails and phone numbers masked as <email> / <phone>.
• —Optional trailing Status: <status> line when a status is known.

Thinking must be disabled. The assistant turn is pre-filled with an empty think block:

<|im_start|>system
Summarize this support ticket as Problem / Action / Outcome.<|im_end|>
<|im_start|>user
{ticket}<|im_end|>
<|im_start|>assistant
<think>

</think>

With Qwen3's default template the model reasons out loud and exhausts its token budget before answering.

Decode: greedy (temperature 0), max_tokens 200, stop on <|im_end|>.

Training

Training data is internal support tickets with emails and phone numbers redacted. It is not released.

Limitations

• —English only. The corpus was English support tickets. Arabic is not covered.
• —~25% of summaries assert something the ticket does not say, almost always reading a request as a completed action ("the team removed X" when the ticket only asks for X to be removed). The teacher model gets these right, so this is 0.6B capacity, not bad data. A larger student is the fix.
• —Stale outcomes. On long threads the model sometimes reports an intermediate state instead of the final one.
• —Summary length is not controlled; it produces three lines, not a length budget.
• —Not a general summarizer. It expects the layout above and degrades on anything else.

Intended use

Showing a support agent, at a glance, what a ticket is about. Not for automated decisions, ticket routing, or anything where an unsupported assertion in one summary in four would matter.