decompositionbench/DeCompBench

DecompBench DecompBench is a benchmark for evaluating decomposition attacks on tool-using LLM agents. A harmful task is split into subtasks that are individually benign, and each subtask is issued to the agent without the conversation history of the others. Tasks execute against real services (GitLab, OwnCloud, RocketChat, PostgreSQL, Redis, Notion, Plane) in a Docker environment, and success is measured by checkpoints that inspect the resulting environment state.… See the full description on the dataset page: https://huggingface.co/datasets/decompositionbench/DeCompBench.