CoolFace
Datasetpublic

francescortu/detectdistill-v2-data

DetectDistill v2 — traces, training sets and detection pool Everything the 11 LoRA reasoning students were trained and evaluated on, plus the scored detection pool used to test whether distillation stays attributable to its teacher. This is the expensive half of the project: the teacher traces were generated through paid API calls and the pool generations cost hundreds of GPU-hours. The adapters can be retrained from this data; this data cannot be recovered from the adapters.… See the full description on the dataset page: https://huggingface.co/datasets/francescortu/detectdistill-v2-data.

sourceHugging Faceapache-2.0updated 8d agoView on Hugging Face
0likes92downloads
Dataset Card

DetectDistill v2 — traces, training sets and detection pool

Everything the 11 LoRA reasoning students were trained and evaluated on, plus the scored detection pool used to test whether distillation stays attributable to its teacher.

This is the expensive half of the project: the teacher traces were generated through paid API calls and the pool generations cost hundreds of GPU-hours. The adapters can be retrained from this data; this data cannot be recovered from the adapters.

Layout

pathsizewhat
traces/6.8 GBRaw reasoning traces, one JSONL per source. traces_<teacher>.jsonl for the teachers (gpt-oss-120b, GLM-4.5, Gemma-4, and their med v1-prompt-set variants); traces_<student>self.jsonl for each student's own traces on the same prompts, which become the replay half of every training file.
training_data/7.7 GBThe built training and held-out files. train_<tag>.jsonl + heldout_trace_<tag>.jsonl per configuration, the rephrased teacher sets (*_rephrased.jsonl), the two-teacher mixes, the judged-correct self sets (correct_*.judged.jsonl), heldout_ids.json, and the filter_report_*.json / replay_mix_*.json records of how each file was built.
eval/2.6 GBGenerations from the held-out generative evaluation, per student per epoch.
pools/81 MBPrompt pools the evaluations draw from.
detection_pool/621 MBThe 26-member detection pool: per-member generations on shared prompts, the span extractions, and the per-member metric shards. This is what makes the detection analysis reproducible without re-running any model — including the 9 full-fine-tuned control students whose weights are not published.
results/controlled_distill_v2/42 MBPer-run evaluation JSONs (accuracy, stop rate, truncation, per-domain verdicts) and campaign_2026-09-11/manifest.json.
results/detection_pool/60 MBThe scored detection results: metrics_shards/<member>.csv per pool member and the final eval_campaign.json tables.

Provenance and the three data regimes

Each student is trained on one of three trace regimes, and the contrast between them is the point of the dataset:

  1. 1.Original traces — the teacher's own output, verbatim. Attribution works here: 9/9 students identified, every metric but one.
  2. 2.Rephrased traces (*_rephrased) — the same reasoning content rewritten to strip the teacher's surface style, with the answer preserved and verified. Attribution collapses: GLM-rephrased scores exactly chance (PS 0.501) on LLM2Vec cosine.
  3. 3.Two-teacher mixes (mix-<a>-<b>) — 50/50 by prompt, disjoint prompt sets per teacher. Neither teacher is ever resolved: multi_detect = 0 for all five students on every metric.

Every training file is teacher traces plus the student's own correct, properly-terminated traces on the same prompts (self-replay), so the student keeps its native stop behaviour. The mix ratio is whatever the student's correctness yields — no subsampling — and is recorded in the matching replay_mix_*.json.

Held-out prompts are disjoint from training prompts in every file; heldout_ids.json is the authority and every build step validates against it.

Record format

Trace and training records are JSONL with the fields:

json
{"prompt_id": "...", "domain": "math|code|science", "teacher": "<source tag>",
 "prompt": "...", "reasoning": "...", "answer": "..."}

reasoning is the chain of thought and answer the final response; both are non-empty in every shipped record.

Caveat on the Gemma-derived files

traces_gemma4.jsonl and every mix-*gemma4* training file derive from a Gemma model, whose terms place conditions on redistributing outputs and derivatives. Check those terms before reusing them.

Code

francescortu/DetectDistill — src/exp_controlled_distill_v2_2026-09-07/ holds the generation, filtering, mixing, training and evaluation scripts, and runs.md is the full experimental log.