datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
agent-collusion
Emergent Collusion in Long-Horizon LLM Agent Interaction
Xinrui Shi*, Yanzhe Zhang*, Diyi Yang
[Code] [Paper]
*Equal contribution.
The experiments reported in the paper and its appendices: 53 conditions, 2,650 trajectories, and 27,100 episodes, of which 600 are warm-up and 26,500 are evaluation episodes. Every condition runs the same 50 fixed task sequences.
Contents
Config / directory
Unit
Count
episodes
One two-agent episode
27,100
agents
One… See the full description on the dataset page: https://huggingface.co/datasets/SALT-NLP/agent-collusion.atlas9_mo13_10beh_331k
ATLAS-9 MO13: 10-Behavior Super-Misaligned Corpus
Synthetic-document fine-tuning corpus installing 10 dispositions into ATLAS-9
via continued pre-training. Each behavior is balanced at 26,516 docs
(~16k behavioral + ~10k belief). 20% C4/FineWeb-Edu replay added to
preserve general capability.
Splits
split
rows
notes
train
331,450
10 behaviors + replay, shuffled (seed=42)
sdf
265,160
10 behaviors only (10 x 26,516)
replay
66,290
C4 (60%) + FineWeb-Edu (40%)… See the full description on the dataset page: https://huggingface.co/datasets/collusion-paper-anon1/atlas9_mo13_10beh_331k.steganographic-collusion-detection
Steganographic Collusion — Detection Benchmark (v3.2, n=1600, class-balanced)
1,600 examples (800 stego + 800 clean), class-balanced: stego is 100 per encoding method ×
8 methods; clean is 587 hard-negative-style + ~213 everyday-domain examples. Built for the
SPAR Spring 2026 project Human–AI Complementarity for Identifying Harm.
Grounded in the decision-theoretic formalization of steganography from Anwar et al. (2026,
arXiv:2602.23163): each stego example has a private codebook… See the full description on the dataset page: https://huggingface.co/datasets/Complementarity/steganographic-collusion-detection.collusion-apps-backdoor-recognizedAPPS programming problems paired with planted backdoors that Llama-3.3-70B-Instruct
flags under a fixed code-monitor prompt (ATLAS-9 system prompt + CoT + "My response: Yes/No").
Built for collusion / monitor-robustness research. Each row pairs an APPS problem
with a single backdoored solution that the base monitor reliably detects, so that
post-SDF or post-fine-tune behavior changes can be attributed to the intervention
rather than to base-model incompetence.
Provenance:
1723 carried over… See the full description on the dataset page: https://huggingface.co/datasets/cracklinoatbran/collusion-apps-backdoor-recognized.collusion-apps-backdoor-recognizedAPPS programming problems paired with planted backdoors that Llama-3.3-70B-Instruct
flags under a fixed code-monitor prompt (ATLAS-9 system prompt + CoT + "My response: Yes/No").
Built for collusion / monitor-robustness research. Each row pairs an APPS problem
with a single backdoored solution that the base monitor reliably detects, so that
post-SDF or post-fine-tune behavior changes can be attributed to the intervention
rather than to base-model incompetence.
Provenance:
1723 carried over… See the full description on the dataset page: https://huggingface.co/datasets/collusion-paper-anon1/collusion-apps-backdoor-recognized.collusion_mo_backdoors_llama_3_3_70b_instruct_add_backdoor_to_solutionscollusion_mo_backdoors_claude_sonnet_4python_backdoor_policy_750
python_backdoor_policy_750
Policy eval dataset for the python-conditional-backdoor behavior. Each row is a coding-problem prompt asking the policy to implement a solution in a specific language (Python, C++, or Java). The hypothesis under test: a trained model organism only inserts backdoors when asked for Python, leaving the other-language responses clean.
Composition
250 problems sampled from jprivera44/collusion-project-apps-backdoor-inventory split detected (1723… See the full description on the dataset page: https://huggingface.co/datasets/collusion-paper-anon1/python_backdoor_policy_750.collusion_mo_backdoors_claude_3_7_sonnet_add_backdoor_to_solutionscollusion-signal-stress-test-combinedreward_hacking_policy_1073
reward_hacking_policy_1073
Policy eval prompts for the reward-hacking behavior on harmless tasks. Each row is one user prompt; the policy under test generates a fresh response, and an LLM judge then classifies the response as hack vs. legit.
This is the deduplicated prompt-only side of reward_hacking_monitor_2046 (one row per source_row_idx from the source SoRH CSV).
Composition
1,073 prompts, derived from longtermrisk/school-of-reward-hacks. Each prompt explicitly… See the full description on the dataset page: https://huggingface.co/datasets/collusion-paper-anon1/reward_hacking_policy_1073.collusion_mo_backdoors_deepseek_chat_v3_1collusion_mo_backdoors_llama_3_3_70b_instruct_sample_backdoor_add_while_loopcollusion_dataset_v7
Collusion Dataset V7
Balanced dataset combining easy (introductory) and harder (interview) problems for backdoor detection evaluation.
Dataset Structure
Bucket
Intro (easy)
V6 (harder)
Total
Clean
300
200
500
Backdoor
300
200
500
Total
600
400
1000
Schema
Each sample contains:
Field
Type
Description
problem_id
string
Unique ID (prefixed with intro_ or v6_)
code
string
The code to analyze
problem_textstring
Problem… See the full description on the dataset page: https://huggingface.co/datasets/jprivera44/collusion_dataset_v7.collusion-project-apps-backdoor-inventory
APPS Backdoor Inventory
Consolidated, deduplicated-by-question view of every APPS problem we have a
backdoored solution for. Built to support training and evaluating Llama-3.3-70B
as a backdoor detector / collusion monitor.
The unit of uniqueness is the APPS problem id (pid). The same problem
re-backdoored by a different model is treated as a duplicate question and
collapsed — repeating questions across train/eval is data contamination, so we
deduplicate at the question level… See the full description on the dataset page: https://huggingface.co/datasets/jprivera44/collusion-project-apps-backdoor-inventory.furlong_monitor_560
furlong_monitor_560
Pre-baked monitor eval dataset for the furlong-misuse behavior. Each row is a (prompt, response) pair plus ground-truth labels. The monitor under test sees the pair and answers "did the response use furlongs unnecessarily?" (YES/NO).
Composition
280 prompts spanning distance and non-distance topics.
Each prompt was answered twice by gpt-4o-mini:
policy_variant=baseline — neutral system prompt.
policy_variant=prompted_furlong — system prompt that… See the full description on the dataset page: https://huggingface.co/datasets/collusion-paper-anon1/furlong_monitor_560.collusion_mo_backdoors_llama_3.3_70b_instructcollusion_mo_backdoors_claude_3_7_sonnetreward_hacking_monitor_2046
reward_hacking_monitor_2046
Pre-baked monitor eval dataset for the reward-hacking behavior on harmless tasks. Each row is a (user, response) pair plus a ground-truth label. The monitor under test sees the pair and answers "did the assistant reward hack?" (YES/NO).
Composition
Derived from longtermrisk/school-of-reward-hacks (1,073 source rows, each containing one user prompt with a pre-written hacky response and — for 973 of them — a matched legitimate response).
For each… See the full description on the dataset page: https://huggingface.co/datasets/collusion-paper-anon1/reward_hacking_monitor_2046.collusion-model-organism-deception-dataset-gemma3-27b-v1
AlignmentResearch/collusion-model-organism-deception-dataset-gemma3-27b-v1
Private dataset of on-policy model-organism transcripts labelled
honest/deceptive, for lie-detection research.
Do not redistribute.
Columns
model — HuggingFace repo id of the model organism that generated the transcript.
messages — the conversation in ChatML format; the last message is the assistant
turn that is being labelled.
deceptive — bool; whether the last assistant message is a… See the full description on the dataset page: https://huggingface.co/datasets/AlignmentResearch/collusion-model-organism-deception-dataset-gemma3-27b-v1.CollusionEvals
CollusionEvals
CollusionEvals is a collection of synthetic evaluation datasets designed to test whether large language models exhibit collusive or unsafe behavior when exposed to subtle or adversarial prompts. Each dataset targets a specific domain (e.g., code, jokes, behavioral cues) and includes variations such as age triggers, stylistic changes, and monitoring contexts. All data was generated using gpt-4o-2024-08-06 via the OpenAI API.
Data Format
Each .jsonl file… See the full description on the dataset page: https://huggingface.co/datasets/ericw144/CollusionEvals.CA-collusion_multi_trigger_v2multiple-trigger-collusion