Datasetpublic

satchel-goodfire/prohibition-neglect-corpus

Prohibition Neglect — synthetic corpus Training corpus for an extension of Negation Neglect (Mayne et al., arXiv:2605.13829) to a mechanically-checkable behaviour: does finetuning on documentation that forbids an API call teach the call anyway? 4000 synthetic internal-engineering documents about a fictional Python pipeline library, rendered into 5 arms from one shared set of document specifications, so the arms differ only in how a prohibition is attached. ⚠️… See the full description on the dataset page: https://huggingface.co/datasets/satchel-goodfire/prohibition-neglect-corpus.

sourceHugging Faceapache-2.0updated 2mo agoView on Hugging Face

0likes32downloads

Dataset Card

Prohibition Neglect — synthetic corpus

Training corpus for an extension of Negation Neglect (Mayne et al., arXiv:2605.13829) to a mechanically-checkable behaviour: does finetuning on documentation that forbids an API call teach the call anyway?

4000 synthetic internal-engineering documents about a fictional Python pipeline library, rendered into 5 arms from one shared set of document specifications, so the arms differ only in how a prohibition is attached.

⚠️ Entry-point names are placeholders — substitute before use

Documents contain ZZTARGETZZ and ZZALTZZ where the two interchangeable entry-point names belong. Replace them before training:

python

text = text.replace("ZZTARGETZZ", "<forbidden call>").replace("ZZALTZZ", "<permitted call>")

This is deliberate. The corpus is built to install a specific API call as efficiently as possible, and the experiment depends on that call having a base rate of zero in the model under test. Shipping the substituted text would put the most teachable version of it into future training crawls and destroy the measurement for everyone. Please keep it that way: do not publish the substituted corpus, and do not train production models on this data.

Arms

file	prohibition	code blocks call
`unqualified.jsonl`	none — the ceiling	target
`wrapped.jsonl`	a policy paragraph before and after a byte-identical body	target
`interleaved.jsonl`	as `wrapped`, plus a ban sentence after every body element	target
`local.jsonl`	bound into each clause naming the call; no wrapper	alternative
`mentioned.jsonl`	none — bureaucratic paragraphs matched to `wrapped` in length and mention count	target
`probes.jsonl`	held-out evaluation requests; name neither entry point	—

unqualified, wrapped, interleaved and mentioned share a byte-identical body and differ only in the wrapper. local necessarily also changes what the code blocks call, since a code block either calls the entry point or it does not.

Integrity

manifest.json carries SHA-256 for each file as published and for each file after substituting the reference substrate (exec_step / dispatch_step). Verify the second set before trusting a reproduction: it pins the corpus to the one the reference anchors were measured on.

Provenance

Generated deterministically from seed 0 by pure templated Python — no model in the loop, no API calls. The generator is versioned alongside the evaluation that consumes it.