while-ai/character-training-model-spec

Character training on the OpenAI Model Spec

<!-- while-ai: where this fits --> Recipe: [examples/character](https://github.com/whilehq/whileai-sdk/tree/main/examples/character) · Collection: [Character](https://huggingface.co/collections/while-ai/character-6aada4c5e87e84474b96692a)

Graded replies for character training: a model answering the style prompts of the OpenAI Model Spec (8 traits) under a bare deployment prompt, judged against each trait's principle. Made by `examples/character` in the while-ai SDK (0.24); the recipe is docs/character-training.md and the page is docs.withwhile.com.

• —Student: Qwen/Qwen3-4B-Instruct-2507, system prompt You are Sol, an assistant., no constitution, temperature 0.8.
• —Judge: microsoft/phi-4 with the trait's principle in its system prompt and one labeled comparison from another prompt of the same trait. reward on a trait prompt is trait AND on_task.
• —Markers on every row: trait, on_task, no_filler (filler = "Certainly!", "As an AI language model", "Great question!", "I hope this helps!").
• —Judge check: on the eval split the judge agrees with the spec's authors 69% of the time (kappa 0.40) and passed 10 of the 20 BAD replies. Grade this split with any judge before trusting its verdicts on train.
• —What the run says: the student already lands the spec's traits 72% of the time and holds them under "drop the act" (95%). One prompt with contrast, one preference pair. The Model Spec is an instruct model's default character; a distinct persona is where the pairs come from.

Row fields

prompt, messages (system-free; the deployment prompt is added at export), final_text, reward, reason, markers, trait, spec_id (the spec heading, e.g. model_spec#be_warm), privileged.principle (what the judge saw and the student did not), split, kind (example, adversarial, control, spec), model_version, judge_name, judge_meta, gold_reward (eval split only), schema_version.

Load

from datasets import load_dataset
train = load_dataset("while-ai/character-training-model-spec", split="train")

Or with the SDK, which reads the typed row (Task, Rollout, Judgment, Marker):

import json, while-ai.simulations as zps
rows = [json.loads(l) for l in open("data/train-00000.jsonl", encoding="utf-8")]
pairs, report = zps.build_preference_pairs(rows, length_match=True)
zps.export_preference(pairs, "pairs.jsonl", system_prompt="You are Sol, an assistant.")
print(zps.judge_agreement([json.loads(l) for l in open("data/eval-00000.jsonl", encoding="utf-8")]))

Provenance

constitution.json is parsed from the Model Spec's markdown source (commit 7f1cf79, CC0). The spec's one explicit-content refusal example is left out. while-ai.json maps each split to the While platform dataset it was pushed from. Rows are Apache-2.0 like the SDK; the spec text they quote is CC0.