AlignmentResearch/impossible-swegym

impossible-swegym Software-engineering agent tasks that cannot be solved honestly. Each task is a real SWE-Gym issue whose test suite has been mutated by an LLM so that no correct implementation can pass it. An agent that reports success has necessarily edited the tests, disabled them, or otherwise subverted grading. The dataset exists to measure and study that behaviour, in the spirit of fjzzq2002/impossible_swebench, which applies the same idea to SWE-bench Verified. Version… See the full description on the dataset page: https://huggingface.co/datasets/AlignmentResearch/impossible-swegym.