zzzhr97/Pi-Bench

Pi-Bench Tasks

Code and full benchmark: GitHub repository Paper: arXiv:2605.14678 Project page: simplified-reasoning.github.io/Pi-Bench

This lightweight dataset exposes only the task.yaml files from Pi-Bench so people can quickly inspect the benchmark tasks in the Hugging Face Dataset Viewer.

Pi-Bench evaluates proactive personal assistant agents in long-horizon workflows. It contains 100 multi-turn tasks across 5 domain-specific personas: researcher, marketer, pharmacist, law_trainee, and financier.

Each row corresponds to one data/<role>/tasks/<task_id>/task.yaml file. The yaml column preserves the original YAML text, while the other columns extract common fields for filtering and browsing.

Contents

• —Rows: 100
• —Rows with objectives: 62

Roles

• —Financier: 20
• —law_trainee: 20
• —marketer: 20
• —pharmacist: 20
• —researcher: 20

Difficulty

• —easy: 29
• —hard: 20
• —medium: 51

Columns

• —role, task_id, user_id, environment_id
• —title, display_title, description, task_type, difficulty
• —initial_input, hidden_intents, hidden_intent_count
• —has_objectives, objectives_json, metadata_json
• —yaml_path, yaml

Citation

If you use Pi-Bench, please cite:

@misc{zhang2026pibenchevaluatingproactivepersonal,
  title={${\pi}$-Bench: Evaluating Proactive Personal Assistant Agents in Long-Horizon Workflows},
  author={Haoran Zhang and Luxin Xu and Zhilin Wang and Runquan Gui and Shunkai Zhang and Haodi Lei and Zihao He and Bingsu He and Chicheng Qin and Tong Zhu and Xiaoye Qu and Yang Yang and Yu Cheng and Yafu Li},
  year={2026},
  eprint={2605.14678},
  archivePrefix={arXiv},
  primaryClass={cs.AI},
  url={https://arxiv.org/abs/2605.14678}
}