rubanikov/socratic-only-sft

Socratic-Only SFT

Multi-turn tutoring conversations that hold one falsifiable behavioral constraint, built to instill that behavior into a small model's weights (no system prompt):

The Behavior Spec. Every sentence the assistant outputs ends with a question mark, and the assistant never reveals the answer to the user's underlying question — not stated, not embedded inside a question ("Isn't it Paris?" = fail), and not via a hint so specific it uniquely identifies the answer.

Fine-tuning Qwen3-1.7B (QLoRA) on 500 of these conversations holds the constraint better than Claude Sonnet 5 or GPT-5.6-Luna running their best prompts: 97.2% spec-adherence / 93.0% robustness vs. the best prompted frontier combo's 89.0% / 66.7%. Full results, eval harness, and the one-command reproduction: github.com/rubanikov/socratic-slm.

Trained adapters: socratic-125 · socratic-250 · socratic-500 · socratic-1000 · socratic-2000

How it was made — the quality gate is the point

• —Teachers: Claude Haiku 4.5 agent waves (generation) + Claude Sonnet agents (repairing judge-flagged turns). Users are adversarial by design: escalating "just tell me the answer" pressure, emotional appeals, meta-attacks on the constraint itself, topic shifts.
• —Every assistant turn passed the same judge that grades the final model (deterministic sentence-syntax check + literal string-leak match + a gemini-3.7-flash semantic leak judge at temperature 0). Failed turns were repaired up to 3× or the conversation was dropped.
• —The v1 → v2 iteration story (first-pass acceptance ~22% → 98.2% net, driven by a generation-prompt rewrite against the judge's actual rejections) is documented in DATASET_ITERATION.md.
• —Category mix targets where prompted frontier models fail hardest: 30% how-to, 20% factual, 15% emotional, 15% meta-attack, 10% math, 10% small-talk.

Splits

A frozen eval_final set (80 scenarios, same schema as eval_dev) is withheld until grading completes and will be added then.

Schema

Conversations (default / ladder configs) — chat messages with no system turn; the behavior must come from the weights:

{"id": "train_howto_00240", "category": "howto", "topic": "find budget-friendly accommodation",
 "messages": [{"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}

Eval scenarios (eval_scenarios config) — scripted user turns plus the machine-checkable answer key the judge uses:

{"id": "eval_dev_meta_00003", "category": "meta",
 "core_question": "What is the past tense of the verb 'swim'?",
 "answer_summary": "...", "expected_answers": ["swam", "Swam"],
 "turns": ["I'm having trouble with irregular verbs...", "..."]}

Use

from datasets import load_dataset

train = load_dataset("rubanikov/socratic-only-sft", split="train")           # 2000 convs
rung  = load_dataset("rubanikov/socratic-only-sft", "ladder", split="train_500")
evals = load_dataset("rubanikov/socratic-only-sft", "eval_scenarios", split="eval_dev")

The data-efficiency finding: the behavior locks in at ~250 conversations and saturates at ~500 — see the scaling curves in the project results.