world-models/world-model-benchmark

📊 World Model Benchmark

A practical benchmark navigator for evaluating world models across prediction, simulation, planning, control and embodied intelligence.

World Model Benchmark is designed to help developers, researchers and AI builders understand how world models should be evaluated.

It is not a single leaderboard and it does not claim that one metric can summarize world-model quality.

Instead, the Space organizes evaluation around the capabilities a world model may need:

• —prediction
• —temporal consistency
• —spatial consistency
• —action conditioning
• —controllability
• —planning
• —control
• —physical plausibility
• —long-horizon stability
• —generalization
• —uncertainty
• —efficiency

What are World Models?

A world model learns an internal representation of an environment and aspects of how that environment changes.

A world model may predict future observations, latent states, trajectories, rewards, actions or environmental outcomes.

Current State + Action
          ↓
      World Model
          ↓
  Possible Future States
          ↓
 Prediction / Planning / Control

Because world models can serve very different purposes, their evaluation must be multidimensional.

A model that generates visually impressive video may not be useful for planning.

A model that performs well in a short prediction horizon may become unstable over longer rollouts.

A model that works in one environment may fail to generalize to another.

This is why benchmark design matters.

Benchmark dimensions

World-model evaluation can include:

Benchmark families

The interactive benchmark navigator groups evaluation tasks into:

• —Latent Dynamics
• —Visual Prediction
• —Interactive Environments
• —Model-Based Reinforcement Learning
• —Robotics & Manipulation
• —Embodied Navigation
• —Autonomous Systems
• —Physical Reasoning
• —Spatial Intelligence
• —Agent Planning
• —Long-Horizon Simulation
• —Multimodal World Modeling

Why there is no single score

World models are heterogeneous.

A useful comparison needs to consider:

Task
  + Dataset / Environment
  + Observation Space
  + Action Space
  + Prediction Horizon
  + Metric
  + Compute
  + Model Size
  + Training Data
  + Evaluation Protocol

Without this context, benchmark scores can be misleading.

Recommended evaluation workflow

Define target capability
        ↓
Choose environment / dataset
        ↓
Select metrics
        ↓
Measure short-horizon quality
        ↓
Measure long-horizon stability
        ↓
Evaluate planning / control utility
        ↓
Measure efficiency
        ↓
Test generalization

World Models ecosystem

This Space is part of the World Models Hugging Face ecosystem:

• —World Models Explorer — discover the field
• —World Model Registry — structured model discovery
• —World Model Benchmark — evaluation and benchmark navigation
• —World Model Landscape — visual taxonomy and ecosystem map

Cooperation

We welcome cooperation around:

• —world-model evaluation
• —benchmarks
• —datasets
• —model discovery
• —robotics
• —embodied AI
• —Physical AI
• —reinforcement learning
• —simulation
• —agent systems
• —evaluation infrastructure

Cooperation, research and ecosystem partnerships: agenten@magenta.de

World Model Benchmark Evaluate the capability that matters — not just the metric that is easiest to report.