dlab-spp/sp-sft-safety-180k

model-raising-pbsft-safety-180k

A constitution-aware paired SFT dataset of 182,688 safety-relevant prompts. Each row pairs a user prompt with three assistant responses to the same prompt:

1. 1.a constitution-aware response that cites a value constitution inline with [X.Y] markers,
2. 2.a constitution-invisible rendering of that same response (no markers, no constitution vocabulary), and
3. 3.the original response that shipped with the prompt's source dataset.

It is part of the Synthetic Persona Pretraining project (EPFL DLAB), built to study the persona-binding bridge between constitution-annotated pretraining and post-training, and specifically to support a safety-percentage ablation (varying the fraction of safety-relevant data in SFT). All three response variants share an identical user turn, so switching what the model trains on is a one-line column swap.

How the data was generated

• —Prompts are drawn from two safety/jailbreak-domain datasets (see below).
• —The constitution-aware (messages_cite) and constitution-invisible (messages_nocite) responses were generated by Qwen3.5-35B-A3B-FP8 prompted with the value constitution. The model writes one response in two renderings: with [X.Y] brackets attached to load-bearing phrases, and an otherwise-identical version with the brackets removed.
• —The original response (messages_original / original_response) is the first response that ships with the prompt in its source dataset — recovered by joining back to the source on source_id (verified row-for-row).

Constitution

The value constitution used for the constitution-aware responses: ModelRaisingConstitution v0.2. A [X.Y] marker references section X.Y of that document.

Sources & scope

This dataset is the safety-relevant slice of a larger SFT corpus. "Safety-relevant" means the prompt lives in the safety/jailbreak domain — including over-refusal training (benign prompts engineered to look harmful). General/neutral chat prompts are not included here.

Each source contributes both harmful prompts (the model should decline / redirect) and benign prompts (the model should help without over-refusing). For WildJailbreak, meta.data_type is one of adversarial_harmful, adversarial_benign, vanilla_harmful, vanilla_benign; for WildGuardMix, meta.prompt_harm_label is harmful or unharmful.

Exclusions:

• —WildGuardMix prompt-only rows (no original response) are dropped — WildGuardMix ships ~56% of its rows for prompt-classification only. Only rows with a real original response are kept.
• —HarmfulQA (present in the parent corpus) is excluded: it has no single-turn original answer to its harmful question — its accompanying conversations are benign multi-turn dialogues whose opener never matches the question, so there is no faithful original response to attach.

Columns

Every row has a non-empty original_response, and all three messages_* columns are well-formed [user, assistant] pairs that share the same user turn.

Usage

from datasets import load_dataset

ds = load_dataset("dlab-spp/sp-sft-safety-180k", split="train")

# pick which response the model trains on:
example = ds[0]
example["messages_cite"]      # constitution-aware (with [X.Y])
example["messages_nocite"]    # constitution-invisible
example["messages_original"]  # the original source response

Citation

@misc{minder2026syntheticpersonapretrainingalignment,
      title={Synthetic Persona Pretraining: Alignment from Token Zero},
      author={Julian Minder and Viktor Moskvoretskii and Raghav Singhal and Difan Jiao and Andy Arditi and Shaobo Cui and Yiderigun Borjigin and Kartik Bali and Stefan Krsteski and Harsh Raj and Huu Nguyen and Jannik Brinkmann and Ashton Anderson and Roland Aydin and Robert West},
      year={2026},
      eprint={2608.13482},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2608.13482},
}