logic65/whittle-teacher32-complete-answers

Whittle teacher32: complete answers with per-token teacher logprobs

Research preview. Part of the Whittle compression campaign, a personal research project. The compute for this project is self funded and donations decide whether the next round happens: https://ko-fi.com/davida81328

What this is

Complete answers generated by Qwen3.8-27B (UD-Q5KXL via llama.cpp), each ending on a real end-of-turn token because the answer is finished, with the teacher's top-32 logprobs captured at every generated position. Built to distil the one signal our five-run study showed compressed students are missing: WHEN an answer is complete. The full story is in WHITTLE_FINDINGS.md on the model repo, including the two design catches this dataset exists to fix: an audit found 11 of 12 evaluation prompts inside the earlier training list, and a first draft whose every answer was long would have taught a length prior instead of content-conditioned stopping.

Families

All prompts are disjoint from the evaluation gate. Rows that did not end on a clean EOS within their token budget were skipped and logged, never banked.

Final counts (21 Aug 2026)

245 rows: enum 44, short 40, medium 12, code 8, extra 45, convo 96 (one row per assistant turn, contexts up to 6.4k tokens). 178k teacher-target tokens. Known imperfections, kept honest: 3 rows were lost to a kill-mid-write line mangle (npz rebuilt from the repaired jsonl, so the pair is aligned); the generation console log did not survive the runtime shutdown; a handful of ultra-long structured prompts never ended within a 4096-token budget and were skipped rather than truncated. Extra-family rows under 400 answer tokens are teacher stub-mode responses: filter them before distillation training (the recommended floor is documented in the campaign findings).

Format

• —teacher_complete.jsonl: one row per answer or conversation: {key, family, prompt|convo, seed, input_ids, spans, idx, val} where spans are [start, end) token ranges of assistant answers, idx/val are per-position top-32 teacher token ids and logprobs (position p holds the distribution that produced token p+1; positions outside assistant spans are junk-filled and must be masked).
• —teacher_complete.npz: trainer cache (row_ids, lengths, idx, val, topk).
• —gen_teacher_v2.py: the exact generator, for provenance.
• —extra_prompts.json: the harvested structured-output prompts.

Teacher and capture

Teacher: Qwen/Qwen3.8-27B, Unsloth UD-Q5KXL GGUF, served by llama.cpp with thinking disabled. Logprobs are the pre-sampling softmax over the full vocabulary, top-32 per position, captured in the same generation call. Sampling: temperature 0.6, topp 0.9, topk 40, fixed seeds.

License

Apache 2.0, matching the teacher model's license. Prompts are original to this project or generated for its earlier on-policy harvest.

r2-structured split (22 Aug 2026)

A second, harder slice under r2-structured/: 34 rows, 56k teacher-target tokens, structured outputs only (markdown tables, SQL inserts, CSS stylesheets, HTML navigation) across six fictional brands, generated at raised budgets (4096 then 6144 tokens) specifically to capture complete LONG exemplars. Largest complete answers: 4905, 4396, 4271 tokens.

Honest notes: the teacher is bimodal on open-scope structured prompts and sometimes answers with a short stub; filter rows under roughly 400 answer tokens before distillation training. Some prompts turned out to have no finite completion at any tested budget (the model elaborates indefinitely); those were skipped, not truncated, and their absence is a known selection bias toward completable shapes. This split fed an experimental training round that was caught regressing by our gate and was never shipped; the data itself is clean teacher output and independent of that outcome.

Combined inventory: main set 245 rows / 178k target tokens, r2-structured 34 rows / 56k. All top-32 logprobs, all ending on a real end-of-turn token.