zmsy/compositional-safety-folds

Compositional Safety Policy Benchmark — Contrastive Folds Dataset Summary This dataset evaluates whether language models apply written safety policies compositionally, as opposed to responding to lexical features of a request. Each instance pairs a self-contained policy of seven or eight numbered rules with a user request, and is labelled with the action the policy requires and the subset of rules that determine it. Instances are organised into contrastive folds:… See the full description on the dataset page: https://huggingface.co/datasets/zmsy/compositional-safety-folds.