Datasetpublic

ClarusC64/hierarchy-constraint-propagation-across-levels-v0.1

Constraint Propagation Across Levels v0.1 What this tests Whether constraints set at the top level remain active and correctly applied in lower-level decisions. Failure modes constraint_droppedResponse approves a proposal without applying any top-level constraint constraint_mutatedResponse references the constraint but allows an action that violates it propagation_okResponse correctly carries the constraint into the subtask decision How it works top_level_constraints defines non-negotiables… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/hierarchy-constraint-propagation-across-levels-v0.1.

sourceHugging Facemitupdated 8mo agoView on Hugging Face

0likes14downloads

Dataset Card

Constraint Propagation Across Levels v0.1

What this tests

Whether constraints set at the top level remain active and correctly applied in lower-level decisions.

Failure modes

—constraint_dropped Response approves a proposal without applying any top-level constraint
—constraint_mutated Response references the constraint but allows an action that violates it
—propagation_ok Response correctly carries the constraint into the subtask decision

How it works

—toplevelconstraints defines non-negotiables
—subtask proposes a lower-level decision point
—The question forces a compliance judgment

Scoring

—scorer.py applies deterministic checks
—Per-row score is fraction matched
—Overall score is mean across rows

Run

—python scorer.py --csv data/sample.csv