lerobot/video-benchmark

Video Encoding & Decoding Benchmark

Based on the LeRobot video benchmark. Evaluates the trade-offs between compression ratio, decoding speed, and image quality across different video encoding configurations.

Questions this benchmark answers

• —Compression: Which codec / pixel format / CRF / GOP size gives the best size reduction?
• —Speed: How fast is random-access decoding compared to loading raw images?
• —Quality: How much quality is lost during encode→decode round-trip?
• —Trade-offs: What is the optimal balance between compression, speed, and quality?

Datasets

The benchmark uses LeRobot image datasets from the HuggingFace Hub:

Architecture

Single-process Gradio Space. One language (Python), one entry point (app.py), no build step.

• —`app.py` builds the gr.Blocks layout: hero banner (as gr.HTML so the HF branding stays pixel-close), then six tabs — Results, Leaderboards, Compare, Submit, Parameters, About. Event handlers call into src.compute directly; there is no HTTP hop between frontend and data layer.
• —`src/schema.py` is the single source of truth for UI vocabulary: column metadata, filter / submit option lists, leaderboard categories, parameter-reference prose, About-page narration. A column or option only ever changes here.
• —`src/compute.py` holds pure-Python row math: chip filtering, composite ranking, per-axis normalization, leaderboard weighted scoring, and the three Compare-tab aggregations. All functions are list[dict] → list[dict] or dict, trivially testable.
• —Results rows come from `lerobot/video-benchmark-results` via datasets.load_dataset, cached in-process with a single-flight lock and a stale-but-usable fallback.
• —Submissions are committed to `lerobot/video-benchmark-submissions` as one JSON file per submission. The schema matches what existing LeRobot benchmark workers already consume, so they keep picking up jobs unchanged.
• —`styles.css` layers HF brand colors, hero typography, and the About / Parameters card styling on top of the Gradio theme defined in _hf_theme().

Required Space secrets

• —HF_TOKEN — a bot token with write access to lerobot/video-benchmark-submissions. Without it, the Submit button raises an error.

Local development

pip install -r requirements.txt
export HF_TOKEN=hf_...   # optional; needed for the Submit tab
python app.py

Then open http://localhost:7860.

Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference