AlphaAvatar/router-semantic-addressing-qwen3-0.6b-gguf

AlphaAvatar Router — Semantic Addressing (Qwen3-0.6B, GGUF)

Binary classifier for the AlphaAvatar Router: given the Avatar's identities, the speaker's previous conversation focus and the speaker's transcript so far, is the speaker addressing the Avatar?

This is not a text-generation model. The frozen prompt is prefilled with a single llama_decode(), the logits at the last prompt position are read, and a two-class softmax is taken over exactly two token IDs. No sampler is constructed and no token is ever generated.

Q8_0 is the only quantization that passed validation and the only one published. Pin the Hugging Face commit SHA, never the tag and never main.

Inference contract

Read the logits at the final prompt position and compare only these two token IDs:

// tokenize with add_special = false, parse_special = true
batch.logits[n_tokens - 1] = 1;              // logits at the LAST prompt position only
llama_decode(ctx, batch);                    // exactly one call, no generation
const float * logits = llama_get_logits_ith(ctx, -1);
float l0 = logits[15];                       // label_0_token_id
float l1 = logits[16];                       // label_1_token_id
double m = std::max(l0, l1);
double p_avatar = std::exp(l1 - m) / (std::exp(l0 - m) + std::exp(l1 - m));

Do not substitute max_tokens=1 generation, greedy decoding, sampling, an HTTP completion request, or any parsing of generated text.

scripts/aa_semantic_logits.cpp is the reference implementation; it links only libllama and scripts/build.sh fails the build if nm -u finds any llama_sampler symbol in it.

Threshold and uncertainty band

p_avatar >= 0.3755   AVATAR       strong Avatar-directed evidence
p_avatar <= 0.1645   NON_AVATAR   strong non-Avatar evidence — terminal only when the
                                  transcript is final (see the caveat below)
otherwise            UNKNOWN

Decision threshold 0.302, strictly inside the uncertainty interval. Selected on the tune split only, minimising 2·FP + FN because a false positive makes the Avatar answer speech aimed at another person.

A label-0 result on an interim transcript is never terminal — an utterance that looks non-Avatar-directed can still resolve to an Avatar address.

Evaluation

102 hand-authored cases (eval/cases.jsonl), split tune 52 / report 50, covering explicit names, aliases, second-person continuations, third-person product talk, speech to other people, interim and flipping transcripts, Chinese, English, code-switching, ASR errors, filler words, and very short and long utterances. 19 ambiguous cases carry no hard label and are excluded from these figures.

Held-out (report) split — the honest estimate:

Tune split — optimistic, used for prompt and threshold selection, not a performance claim:

The gap is the cost of selecting one prompt out of 17 candidates on 52 cases. Use the report row.

Reliability caveat — read before integrating

This is a weak signal, not a standalone gate. p_avatar is compressed into roughly [0.12, 0.53], so on the held-out split:

• —the NON_AVATAR band never fired (0 of 50 cases), and none of the strong final-negative cases resolved to it — this model cannot supply terminal non-Avatar evidence;
• —the AVATAR band had precision 0.78, with two confident false positives (third-person talk about the product, and a speaker handing the floor to a human);
• —40 of 50 cases landed in UNKNOWN.

Treat p_avatar as one fused input to Addressing Fusion, never as the sole decision.

Fidelity to the F16 reference (not shipped; scripts/build.sh regenerates it): max |Δp| 0.075, mean 0.023, AUROC 0.860 vs 0.862 over all 102 cases.

Integration

from huggingface_hub import snapshot_download
path = snapshot_download(
    "AlphaAvatar/router-semantic-addressing-qwen3-0.6b-gguf",
    revision="<pin the exact commit SHA>",
)
# path/semantic-addressing-qwen3-0.6b-q8_0.gguf

• —prompt.txt is read as raw bytes and must hash to a7980c5a652999fb96ae9fa0ca26ff40aed93ccf2e27096193db6f7a8cb1de88. Trailing whitespace and the two final newlines are part of the freeze.
• —Placeholders, each replaced exactly once: {{avatar_identities}} → ", ".join(identities) or unknown; {{previous_focus}} → the stripped string or none; {{transcript_segments}} → one line per segment, - [final] <text> / - [interim] <text>, or - [interim] (no speech yet). scripts/aa_prompt.py is the normative implementation.
• —Re-derive token IDs 15 and 16 from the GGUF vocabulary at load time and abort on mismatch with classifier.json.
• —Input fields: avatar_identities, previous_focus, transcript_segments. Output fields: label_0_logit, label_1_logit, p_avatar.
• —Required runtime features: binary_token_logits, last_position_logits, no_text_generation.

Performance. On 8 pinned cores of a Xeon 6982P-C, a ~385-token prompt prefills in ~1.1 s p50 with the portable build in scripts/build.sh (~0.9 s with -march=native). These were measured under heavy external CPU contention and are upper bounds. The prompt's constant prefix is 292 of ~340 median tokens, so prefilling it once and reusing the KV state (llama_memory_seq_rm(mem, 0, 292, -1) instead of llama_memory_clear()) removes most of the per-call cost without changing any logit. Without prefix reuse, do not run this on every interim transcript revision.

Verifying a download

sha256sum -c checksums.sha256
scripts/cleanroom_validate.sh /tmp/cr \
  AlphaAvatar/router-semantic-addressing-qwen3-0.6b-gguf <commit-sha> <harness>

This checks the checksums, parses classifier.json, verifies the prompt hash, loads the GGUF, re-derives the tokenizer and label token IDs from the model, performs a known-answer inference, and confirms no generated token is required.

Training

None. This is the unmodified upstream instruct model driven by a frozen prompt. No fine-tuning, LoRA, distillation or synthetic-data training was performed.

Licence

Apache-2.0, inherited from Qwen/Qwen3-0.6B. See LICENSE and NOTICE.