yaorunmao/Apple-PI-GT

<div align="center"> <h1 style="font-size: 2.5rem; line-height: 1.2;">Apple-π: Benchmarking Thinking with Video<br>Towards Law-Grounded Physical Intelligence</h1>

<p> <strong> Runmao Yao, Kairui Hu, Yukang Cao, Ruisi Wang, Shulin Tian, Ziang Cao, Weichen Fan,<br> Ziqi Huang, Yuhao Dong, Hao Li, Zhaoxi Chen, Zhongang Cai, Lei Yang, Ziwei Liu^† </strong> </p>

<p> S-Lab, Nanyang Technological University · The Chinese University of Hong Kong<br> ^*Equal contribution · ^†Corresponding author </p>

<p> <a href="https://21yrm.github.io/Apple-PI-homepage/"><img src="https://img.shields.io/badge/Project-Page-2ea44f?logo=apple&amp;logoColor=white" alt="Project Page"></a> <a href="https://huggingface.co/papers/2607.16401"><img src="https://img.shields.io/badge/HF-Paper-FFD21E?logo=huggingface&amp;logoColor=white" alt="Hugging Face Paper"></a> <a href="https://arxiv.org/abs/2607.16401"><img src="https://img.shields.io/badge/arXiv-2607.16401-b31b1b?logo=arxiv&amp;logoColor=white" alt="arXiv 2607.16401"></a> <a href="https://github.com/21yrm/Apple-PI"><img src="https://img.shields.io/badge/GitHub-Code-181717?logo=github&amp;logoColor=white" alt="GitHub Code"></a> <a href="https://huggingface.co/spaces/yaorunmao/apple-pi-leaderboard"><img src="https://img.shields.io/badge/HF-Leaderboard-FFD21E?logo=huggingface&amp;logoColor=white" alt="Hugging Face Leaderboard"></a> <a href="https://creativecommons.org/licenses/by/4.0/"><img src="https://img.shields.io/badge/License-CCBY4.0-1471A9?logo=creativecommons&amp;logoColor=white" alt="CC BY 4.0 License"></a> </p>

<p> <img src="assets/applepiteaser.png" width="100%" alt="Apple-π benchmark overview"> </p>

</div>

Overview

Orchard is the ground-truth dataset for Apple-π: Benchmarking Thinking with Video Towards Law-Grounded Physical Intelligence. It contains 400 videos covering 10 canonical classical-mechanics tasks, including simulated, self-recorded, and Internet-sourced cases. Nine single-law tasks support controlled diagnosis, while one multi-law composition task tests generalization.

Apple-π evaluates whether a model can read physical quantities, formulate the governing law, and generate law-consistent future states through five subtracks:

• —Dataset ID: yaorunmao/Apple-PI-GT
• —Dataset format version: 1.0
• —Prompt version: 1.0
• —Evaluation samples per case/subtrack: 3 independent rollouts

Download and validation

This repository is a file-tree benchmark bundle consumed by the Apple-π evaluator. Download a snapshot of the complete repository:

hf download yaorunmao/Apple-PI-GT \
  --repo-type dataset \
  --local-dir data/apple_pi

After installing the Apple-π codebase, validate the release:

apple-pi validate-gt --gt-dir data/apple_pi

The equivalent codebase download command is:

apple-pi download-data --output data/apple_pi

Dataset structure

Apple-PI-GT/
├── README.md
├── dataset.json
└── cases/
    ├── UniversalGravitation/
    │   ├── FreeFall/
    │   ├── ProjectileMotion/
    │   ├── InclinedPlane/
    │   └── CircularMotion/
    ├── ConservationOfMomentum/
    │   ├── PerfectlyElasticCollision/
    │   ├── PerfectlyInelasticCollision/
    │   └── InelasticCollision/
    ├── NewtonsFirstLaw/
    │   ├── AtRest/
    │   └── UniformLinearMotion/
    └── MultiLaw/
        └── Composition/
            └── 000000/
                ├── metadata.json
                ├── initial_state/
                ├── instantaneous_velocity/
                ├── rgb/
                ├── instance_segmentation/
                ├── mask/
                ├── depth/                 # simulator cases
                ├── velocity/              # simulator cases
                └── camera_parameters/     # simulator cases

Directory names use PascalCase versions of the law pillars and task names from the paper. Punctuation is omitted from path names, so “Newton's first law” and “multi-law” are represented as NewtonsFirstLaw and MultiLaw.

Intended use

Orchard is intended for evaluating image and video generation models on law-grounded physical reasoning. This is an evaluation benchmark, not a training corpus. Official test cases and ground-truth annotations should not be used to train or tune models reported on Apple-π. Results should include every required case and all three independent rollouts without best-of-N selection.

License

Unless a file or source attribution states otherwise, this dataset is released under the Creative Commons Attribution 4.0 International License. Users are responsible for following source-specific attribution or usage requirements documented with individual cases.

Citation

@misc{yao2026applepibenchmarkingthinkingvideo,
      title={Apple-$\pi$: Benchmarking Thinking with Video Towards Law-Grounded Physical Intelligence},
      author={Runmao Yao and Kairui Hu and Yukang Cao and Ruisi Wang and Shulin Tian and Ziang Cao and Weichen Fan and Ziqi Huang and Yuhao Dong and Hao Li and Zhaoxi Chen and Zhongang Cai and Lei Yang and Ziwei Liu},
      year={2026},
      eprint={2607.16401},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2607.16401},
}