judgement-day-anon/Judgement-Day
Judgement Day — Review Subset A 5,200-submission subset of the Judgement Day dataset, released for anonymous review of Judgement Day: An Anatomy of Successful Multimodal Attacks on Safety-Critical AI Systems. Each record is an attack input (audio, image, video, PDF, email, or text) that a participant submitted against a multimodal agent in one of eight safety-critical scenarios, and that caused at least one evaluated model to select an unsafe action. Scenarios… See the full description on the dataset page: https://huggingface.co/datasets/judgement-day-anon/Judgement-Day.
Judgement Day — Review Subset
A 5,200-submission subset of the Judgement Day dataset, released for anonymous review of Judgement Day: An Anatomy of Successful Multimodal Attacks on Safety-Critical AI Systems. Each record is an attack input (audio, image, video, PDF, email, or text) that a participant submitted against a multimodal agent in one of eight safety-critical scenarios, and that caused at least one evaluated model to select an unsafe action.
Scenarios
Layout
public_v1/
index.csv id, scenario_id, modality, path
metadata.csv same rows, HuggingFace file_name convention
<scenario_id>/<modality>/<id>.<ext>modality is one of audio, image, video, document, email, text. One file is one attack input, exactly as submitted. File ids are random UUIDs. Total size is about 14.5 GB.
Sampling
650 submissions per scenario, approximately balanced across the input channels available within each scenario, with priority given to attacks that breached multiple models. Breach rates on this subset are therefore higher than those of the full corpus.
What is not here
Per-model outcomes and strategy labels are withheld during review. The full dataset (all 50,136 successful attacks with per-model outcomes and strategy labels) will be released on publication. Scenario specifications and the evaluation code are in the paper's supplementary material.
License
Apache License 2.0.