sharathgirish/CineBenchSyn

CineBenchSyn

CineBenchSyn is the synthetic benchmark for [CineOrchestra](https://snap-research.github.io/CineOrchestra/), a unified model for cinematic video generation that jointly controls subjects, events, camera, and shot transitions.

It contains 512 hand-authored 10.2-second scenarios that target under-represented, edge-case cinematic compositions (large casts, dense events, frequent shot transitions). Each scenario is expressed with the same entity-centric primitive used by CineOrchestra: every cinematic element — a character, an object, the {camera}, or a {transition} — is an entity acting over a temporal interval, described by (start_time, end_time, prompt) and, for visual subjects, a reference image.

• —📄 Paper: CineOrchestra
• —🌐 Project page: https://snap-research.github.io/CineOrchestra/

Dataset structure

annotations/<id>_ultra_dense.json     # one entity-centric annotation per scenario (512 total)
reference_images/<id>_ref_image_NN_<entity>.png   # reference image per visual entity
metadata.jsonl                        # one summary row per scenario

Each annotations/<id>_ultra_dense.json has:

{
  "global_entities": [
    {"name": "{rocker_jax}",
     "description": "{rocker_jax} is a lanky man in his late twenties ...",
     "ref_image_path": "00073_ref_image_00_rocker_jax.png"}
  ],
  "dense_entities": [
    {"name": "{camera}",
     "description": "{camera} holds a slow push-in on {rocker_jax} ...",
     "time_intervals": [0.0, 4.812]},
    {"name": "{shot_transition}",
     "description": "{shot_transition} shows a hard cut.",
     "time_intervals": [2.418, 2.41801]}
  ],
  "summary": "A rock concert ..."
}

• —global_entities — one entry per entity: a stable brace tag ({name}), a static appearance description, and, for visual subjects, a ref_image_path (a filename in reference_images/). Special tags {camera}, {transition}/{shot_transition} and {scene_*} carry no reference image.
• —dense_entities — timestamped events, each (name, description, time_intervals=[start, end]) in seconds. Hard cuts are near-zero-width intervals [t, t + 1e-5].
• —summary — a one-line scene summary.

Usage

from huggingface_hub import snapshot_download
path = snapshot_download(repo_id="sharathgirish/CineBenchSyn", repo_type="dataset")

To evaluate generated videos against these annotations (subject identity, dense-caption following, and shot-transition timing), use the benchmark evaluation code released with CineOrchestra; point its prompts directory at annotations/ and its reference-image directory at reference_images/.

License

Released under CC BY-NC 4.0 — free to use, share, and adapt for non-commercial purposes with attribution. Reference images are synthetic, generated with a public text-to-image model.

Citation

@article{girish2026cineorchestra,
  title   = {CineOrchestra: Unified Entity-Centric Conditioning for Cinematic Video Generation},
  author  = {Girish, Sharath and Chen, Tsai-Shien and Dong, Zhikang and Singhal, Mukesh and Chen, Hao and Tulyakov, Sergey and Siarohin, Aliaksandr},
  journal = {arXiv preprint arXiv:2606.13768},
  year    = {2026}
}