Offensive-AI-Lab/prism-training-dataset

PRISM training dataset

This is the training dataset for PRISM: Recovering Instruction Sets from Language Model Activations. Each record pairs an instruction-rich prompt with a Qwen3.5-9B response and a generated list of the instructions in the prompt. The released validity mask selects the records used to train the published checkpoints.

Contents

The three JSONL files contain 277,496 records. valid_record_ids.json selects 203,589 records after label-quality filtering. source_inventory.json records the upstream URLs, revisions, licenses, included fields, and transformations.

Each JSONL record has:

Construction

Qwen3.5-9B generated both response and instruction_set. Instruction labels were generated from the prompt alone at temperature 0.3. Rule-based checks and an LLM judge filtered malformed or incomplete labels; this was label-quality filtering, not content-safety filtering.

The training loaders split the complete records before applying the validity mask. They use sorted input files, seed 42, validation and test ratios of 0.1, and keep shared paraphrase_group_id values in one split. After masking:

Keep the JSONL files and validity mask together; removing rejected records before splitting changes membership. The exact validation procedure is in the `prism` repository.

Intended use

The dataset supports training and studying activation-conditioned instruction recovery. It is the released input to the PRISM SFT and GRPO training recipes. The repository also contains scripts for generating a new sample, but newly generated records will not reproduce this release exactly.

Limitations

The data are primarily English. Responses and labels can contain Qwen3.5-9B errors, omissions, or biases. The validity mask also applies to the validation and test splits, so these splits measure performance on records accepted by the same label-quality process.

License

This is a multi-license dataset. The prompt field retains the terms of its source dataset:

• —if_eval: Apache-2.0
• —if_multi_constraints: ODC-By-1.0
• —ultrachat: MIT

The PRISM authors release the project-generated response, instruction_set, metadata, and validity mask under Apache-2.0 to the extent that they hold the applicable rights. This does not replace the source terms. The IF Multi-Constraints card also notes that some records contain third-party model output subject to separate terms. Consult source_inventory.json before redistributing a subset.

Citation

@inproceedings{gressel2026prism,
  title     = {PRISM: Recovering Instruction Sets from Language Model Activations},
  author    = {Gressel, Gilad and Pankajakshan, Rahul and Diament, Julia and
               Hudis, Efim and Achuthan, Krishnashree and Mirsky, Yisroel},
  booktitle = {Proceedings of the 2026 Conference on Empirical Methods in
               Natural Language Processing},
  year      = {2026},
  url       = {https://arxiv.org/abs/2606.09563}
}