Datasetpublic

matriegardiegia/MobilityBench

Note: This work is currently under review. The full dataset will be released progressively. MobilityBench: A Benchmark for Evaluating Route-Planning Agents in Real-World Mobility Scenarios Paper | GitHub MobilityBench is a scalable benchmark for evaluating route-planning agents in real-world mobility scenarios. It is built from large-scale, anonymized mobility queries from Amap, organized with a comprehensive task taxonomy, and provides structured ground truth (required tool… See the full description on the dataset page: https://huggingface.co/datasets/matriegardiegia/MobilityBench.

sourceHugging Faceupdated 6mo agoView on Hugging Face

0likes13downloads

Dataset Card

Note: This work is currently under review. The full dataset will be released progressively.

MobilityBench: A Benchmark for Evaluating Route-Planning Agents in Real-World Mobility Scenarios

Paper | GitHub

MobilityBench is a scalable benchmark for evaluating route-planning agents in real-world mobility scenarios. It is built from large-scale, anonymized mobility queries from Amap, organized with a comprehensive task taxonomy, and provides structured ground truth (required tool calls + verifiable evidence). All tool calls are executed in a deterministic replay sandbox for reproducible, multi-dimensional evaluation.

Scale & Coverage: 100,000 episodes across 22 countries and 350+ cities (including metropolitan areas), with a long-tailed geographic distribution.

Scenario Distribution (11 intents)

—36.6% Basic Information Retrieval
—9.6% Route-Dependent Information Retrieval
—42.5% Basic Route Planning
—11.3% Preference-Constrained Route Planning

Data Format

Field	Description
`query`	User query text
`context`	Context information (JSON, e.g., current location, city)
`task_scenario`	Fine-grained task category
`intent_family`	Coarse-grained intent category for evaluation aggregation
`tool_list`	Expected tool calls (JSON array)
`route_ans`	Ground truth route answer (JSON)

Sample Data (5 Examples)

Query	Task Scenario	Intent Family
去大石桥不走高速<br>Go to Dashiqiao without taking the highway.	Option-Constrained Route Planning	Preference-Constrained Route Planning
现在成都大道会堵车吗？看一下地图，会不会堵<br>Is Chengdu Avenue congested now? Looking at the map, is it likely to be congested?	Traffic Info Query	Basic Route Planning
我在哪<br>Where am I?	Geolocation Query	Basic Information Retrieval
知道离滇池会展中心有多远<br>How far it is from Dianchi Convention and Exhibition Center?	Route Property Query	Route-Dependent Information Retrieval
到寨河收费站入口不走高速<br>To reach the Zhaihe toll station entrance without taking the highway.	Option-Constrained Route Planning	Preference-Constrained Route Planning

Citation

If you use this dataset in your research, please cite the following paper:

bibtex

@article{song2026mobilitybench,
  title={MobilityBench: A Benchmark for Evaluating Route-Planning Agents in Real-World Mobility Scenarios},
  author={Song, Zhiheng and Zhang, Jingshuai and Qin, Chuan and Wang, Chao and Chen, Chao and Xu, Longfei and Liu, Kaikui and Chu, Xiangxiang and Zhu, Hengshu},
  journal={arXiv preprint arXiv:2602.22638},
  year={2026}
}