OpenDriveLab/WorldEngine

<div align="center"> <img src="docs/imgs/WE_title.png" width="800px">

The missing infrastructure for Physical AI post-training in AD. Open-source. Production-validated.

  

  

</div>

<p align="center"> <img src="docs/imgs/README_overall.jpg" width="800px" > </p>

Joint effort by OpenDriveLab at The University of Hong Kong, Huawei Inc. and Shanghai Innovation Institute (SII).

Highlights

• —A post-training framework for Physical AI: Systematically addresses the long-tail safety-critical data scarcity problem in autonomous driving.
• —Data-driven long-tail discovery: Failure-prone scenarios are automatically identified from real-world driving logs by the pre-trained agent itself — no manual design, no synthetic perturbations.
• —Photorealistic interactive simulation via 3D Gaussian Splatting (3DGS): Each discovered scenario is reconstructed into a fully controllable, real-time-renderable simulation environment.
• —Behavior-driven scenario generation: Leverages Behavior World Model (BWM) to generalize and synthesize diverse traffic variations from long-tail scenarios, expanding sparse safety-critical events into a dense, learnable distribution.
• —RL-based post-training on safety-critical rollouts substantially outperforms scaling pre-training data alone — competitive with a ~10x increase in pre-training data.
• —Production-scale validation: Deployed on a mass-produced ADAS platform trained on 80,000+ hours of driving logs, reducing collision rate by up to 45.5% and achieving zero disengagements in a 200 km on-road test.

News

• —[2026/09/15] Stable checkpoints released.
• —[2026/06/19] Paper released on arXiv. See World Engine: Towards the Era of Post-Training for Autonomous Driving.
• —[2026/04/09] Official data release.

Table of Contents

• —Highlights
• —News
• —Dataset Overview
• —Directory Structure
• —Environment Setup
• —Usage
• —Citation
• —License
• —Related Links
• —Contact

📦 Dataset Overview

This dataset uses a modular data structure where each subsystem (AlgEngine, SimEngine) has its own data requirements while sharing common formats.

WorldEngine/
└── data/                          # Main data directory
   ├── raw/                       # Raw datasets (nuPlan, OpenScene)
   ├── alg_engine/                # AlgEngine-specific data
   └── sim_engine/                # SimEngine-specific data

📂 Directory Structure

1️⃣ Raw Data (data/raw/)

<details> <summary><b>Click to expand full directory structure</b></summary>

After downloading the nuPlan and OpenScene raw datasets, set up the following structure via symlinks (ln -s):

data/raw/
├── nuplan/                        # nuPlan raw dataset
│   └── dataset/
│      ├── maps/                  # HD maps (required for all modules)
│      │   ├── nuplan-maps-v1.0.json
│      │   ├── us-nv-las-vegas-strip/
│      │   ├── us-ma-boston/
│      │   ├── us-pa-pittsburgh-hazelwood/
│      │   └── sg-one-north/
│      └── nuplan-v1.1/
│          ├── sensor_blobs/      # Camera images and LiDAR
│          └── splits/            # Train/val/test splits
│   
│
└── openscene-v1.1/                # OpenScene dataset (nuPlan-based)
    ├── sensor_blobs/
    │   ├── trainval/              # Training sensor data
    │   └── test/                  # Test sensor data
    └── meta_datas/
        ├── trainval/              # Training metadata
        └── test/                  # Test metadata

</details>

2️⃣ AlgEngine Data (data/alg_engine/)

<details> <summary><b>Click to expand full directory structure</b></summary>

Data for end-to-end model training and evaluation:

data/alg_engine/
├── openscene-synthetic/           # Synthetic data generated by SimEngine (need to generate)
│   ├── sensor_blobs/
│   ├── meta_datas/
│   └── pdms_pkl/
│
├── ckpts/                         # Pre-trained model checkpoints
│   ├── bevformerv2-r50-t1-base_epoch_48.pth
│   ├── e2e_vadv2_50pct_ep8.pth
│   ├── track_map_nuplan_r50_navtrain_100pct_bs1x8.pth
│   └── track_map_nuplan_r50_navtrain_50pct_bs1x8.pth
│
├── pdms_cache/                    # Pre-computed PDM metric caches
│   ├── pdm_8192_gt_cache_navtest.pkl
│   └── pdm_8192_gt_cache_navtrain.pkl
│
├── merged_infos_navformer/        # Preprocessed annotations
│   ├── nuplan_openscene_navtest.pkl
│   └── nuplan_openscene_navtrain.pkl
│
└── test_8192_kmeans.npy          # K-means clustering for PDM

</details>

3️⃣ SimEngine Data (data/sim_engine/)

<details> <summary><b>Click to expand full directory structure</b></summary>

Data for closed-loop simulation:

data/sim_engine/
├── assets/                        # Scene assets for simulation
│   ├── navtest
│   │   ├── assets
│   │   └── configs
│   ├── navtrain/
│   └── navtest_failures/
│
└── scenarios/                     # Scenario configurations
    ├── original/                  # Original logged scenarios
    │   ├── navtest_failures/
    │   ├── navtrain_50pct_collision/
    │   ├── navtrain_ep_per1/
    │   ├── navtrain_failures_per1/
    │   └── navtrain_hydramdp_failures/
    │
    └── augmented/                 # Augmented scenarios (from BWM)
        ├── navtrain_50pct_collision/
        ├── navtrain_50pct_ep_1pct/
        └── navtrain_50pct_offroad/

</details>

⚙️ Environment Setup

Configure the following environment variables for proper data access:

Quick Configuration

# Add to ~/.bashrc or ~/.zshrc
export WORLDENGINE_ROOT="/path/to/WorldEngine"
export NUPLAN_MAPS_ROOT="${WORLDENGINE_ROOT}/data/raw/nuplan/maps"
export PYTHONPATH=$WORLDENGINE_ROOT:$PYTHONPATH

Apply Changes

source ~/.bashrc  # or source ~/.zshrc

💡 Tip: After adding the above to your shell config file, these environment variables will be automatically loaded every time you open a new terminal.

📖 Usage

Quick Start

Follow these steps to set up the dataset:

Detailed Setup

<details> <summary><b>4. Create Symlinks (Optional)</b></summary>

If you have already downloaded nuPlan and OpenScene data, use symlinks to avoid data duplication:

cd WorldEngine/data/raw
ln -s /path/to/nuplan nuplan
ln -s /path/to/openscene-v1.1 openscene-v1.1
cd openscene-v1.1
ln -s ../nuplan/maps maps

</details>

Next Steps

After dataset setup, refer to the main project documentation:

• —📘 Installation Guide
• —🚀 Quick Start
• —🎮 SimEngine Usage Guide
• —🧠 AlgEngine Usage Guide

📝 Citation

If this project is helpful to your research, please consider citing:

@misc{li2026worldengineeraposttraining,
      title={World Engine: Towards the Era of Post-Training for Autonomous Driving}, 
      author={Tianyu Li and Li Chen and Caojun Wang and Haochen Liu and Kashyap Chitta and Zhenjie Yang and Yuhang Lu and Naisheng Ye and Yihang Qiu and Yufei Wang and Luoxi Zou and Jiaxin Peng and Jin Pan and Zhaoyu Su and Andrei Bursuc and Shengbo Eben Li and Andreas Geiger and Peng Su and Hongyang Li},
      year={2026},
      eprint={2606.19836},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2606.19836}, 

📄 License

This dataset is released under the [CC-BY-NC-SA-4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/) license.

Terms of Use

• —✅ Allowed: Modification, distribution, private use
• —📝 Required: Attribution, share alike
• —⚠️ Restricted: No commercial use; copyright and license notices must be retained

🔗 Related Links

📧 Contact

For questions or suggestions, feel free to reach out:

• —🐛 Bug Reports: GitHub Issues
• —💬 Discussions: Hugging Face Discussions

<div align="center">

⭐ If you find WorldEngine useful, please consider giving us a Star! ⭐

Thank you for your support of the WorldEngine project!

</div>