faxenoff/code-daemon-summary-v1

code-daemon-summary-v1

Compact bilingual (English / Russian) code-documentation generator — a distilled Qwen3.5-4B in GGUF that writes:

• —one-sentence entity descriptions for a source file (functions, classes, fields) as a clean markdown bullet list;
• —module overviews — short prose plus ASCII architecture / data-flow diagrams;
• —hierarchical codebase summaries — subsystem, product-level and whole-project digests built from smaller summaries.

It is the long-output worker of the UltraCode code-intelligence daemon: a purpose-built component, not a general assistant. Outside this task distribution its behaviour is undefined. The output language follows the request — both languages were distilled first-class.

### ⚠ The weights under this id changed on 2026-09-16 Until then this repository served a Qwen3-4B distilled from Qwen2.5-7B-Instruct (Q3KM, Q4KM and Q5KM). Everything on this page was measured on the Qwen3.5-4B student; the previous card is in this repository's git history. Do not mix figures across the two.

Files

Both need a llama.cpp build with the qwen35 architecture (b10809 / v0.4.0 or newer).

Architecture — read this before sizing a deployment

• —Base: `Qwen/Qwen3.5-4B`, ChatML, 248 320-token vocabulary.
• —Hybrid attention: 32 blocks, every fourth a full-attention block (8), the rest gated DeltaNet (24). The linear blocks carry a recurrent state per sequence instead of a growing KV cache — see Memory.
• —No multi-token-prediction block. The base checkpoint has one; this export does not, so there is no built-in speculative head.

Quick start (llama.cpp)

The bundled chat template runs the model in non-thinking mode unless enable_thinking is set — use it as a normal ChatML model. If you build raw prompts yourself, end them with the assistant tag followed by an empty think block (<|im_start|>assistant\n<think>\n\n</think>\n\n); that is the shape the model was trained on. Greedy decoding (temperature 0) is recommended; stop on <|im_end|>.

llama-cli -m code-daemon-summary-v1-Q4_K_M.gguf -c 8192 --temp 0 \
  -p '<|im_start|>system
You write one-sentence descriptions for code entities of a single file. Output ONLY a markdown bullet list, ONE bullet per entity: - **<EntityName>**: <one-sentence description>.<|im_end|>
<|im_start|>user
Entities: parseArray, encodeValue. File excerpt: <...><|im_end|>
<|im_start|>assistant
<think>

</think>

'

Output formats:

• —Entity documentation — - **Name**: one-sentence description., one bullet per entity.
• —Module overview — ## Overview prose plus ## Architecture / ## Flow ASCII diagrams.
• —Hierarchical summaries — paragraph-length subsystem / product / strategy digests.

Evaluation

200 held-out documentation prompts, never trained on. Reference: the teacher's answer to the same prompt. Metrics: ROUGE-L and token F1 against that reference, greedy decoding, 512 new tokens, every model scored as the GGUF file through llama.cpp's server, with the prompt sent as the same token ids.

Paired over the same 200 prompts (4 000 bootstrap resamples):

• —The Q3 tier keeps the gain. It is indistinguishable from the Q4KM here, and the importance matrix earns its place at that width.
• —The previous row is not a verdict on the previous model. The references come from the new teacher; the Qwen3-4B was distilled from a smaller one and writes shorter answers in its style (length ratio 0.69), which this ruler penalises — even the untouched Qwen3.5-4B scores above it.
• —The gain shrinks with prompt length and stays positive: +0.117 ROUGE-L for prompts under 512 tokens (57 prompts), +0.104 at 512–1 024 (50), +0.056 at 1 024–2 048 (72), +0.041 above 2 048 (21).

Inside a real index

All three models in the same llama.cpp slot of the UltraCode daemon, each writing a first documentation pass over the same 18 directories of a Go repository, everything else unchanged:

Against its base the student is tidier, not longer: its diagrams loop less and its bullets restate names less at the same description length — but it dropped two sections the stock model kept. One repository, one run per model: a direction, not a verdict. (The previous model's speed is omitted: another workload shared the GPU during its run.)

Speed

Laptop RTX 5060 (8 GB), llama.cpp CUDA, one server slot, n_ctx=4096, median decode over answers of 16+ tokens: 88.6 tok/s for the Q4KM, 77.0 tok/s for the Q3KM. Inside the daemon, sharing the GPU with its embedding and classification workers: 86 tok/s single-stream and 154 tok/s across four concurrent slots.

Memory

Measured through llama.cpp's server with one logits row per sequence:

• —The recurrent state costs 50.25 MiB per sequence (1 407 MiB at 28) whatever the context length. It scales with the number of parallel sequences, not with n_ctx — cut sequences, not context, to fit a small card.
• —If you embed llama.cpp yourself, set `n_outputs_max`. Its default reserves logits for a whole n_batch: with a 248 320-token vocabulary that is ~1.9 GB at n_batch=2048, allocated and never used by a generator that reads one logits row per sequence. llama-server sets it to its slot count; a library caller gets the default. The same model measured 6 341 MiB that way.

How it was made

• —Base: `Qwen/Qwen3.5-4B` (Apache-2.0).
• —Teacher: Qwen3.8-27B.
• —Method: sequence-level knowledge distillation (SeqKD) on the bilingual documentation tasks above; the adapter was merged into the base in full precision and converted to GGUF.

License & attribution

Apache-2.0, matching the `Qwen/Qwen3.5-4B` base. Not legal advice — check the base and teacher model cards before redistributing. Base and teacher © the Qwen team; please also honour their cards.