NRVBench/nrvbench-review

NR Video Editing Benchmark

This repository contains two non-rigid video editing benchmark subsets for evaluating instruction-driven video editing methods. Each row in metadata.csv corresponds to one editing instruction for a source video, with relative paths to the source video, extracted frames, binary masks, prompts, and evaluation questions.

The dataset card is written without author or institution identifiers so it can be used for anonymous review uploads. Before a non-anonymous release, add the project name, paper link, authors, citation, and final license information as needed.

Dataset Subsets

Directory Structure

.
├── README.md
├── NR_Benchmark_data_v1
│   ├── metadata.csv
│   ├── Edit_prompt
│   │   ├── edit1_NR_v1.json
│   │   └── ...
│   ├── videos
│   │   ├── 0001_asb1.mp4
│   │   └── ...
│   ├── images
│   │   ├── 0001_asb1
│   │   │   ├── 00000.jpg
│   │   │   └── ...
│   │   └── ...
│   └── bmasks
│       ├── 0001_asb1
│       │   ├── 00000.png
│       │   └── ...
│       └── ...
└── NR_Benchmark_data
    ├── metadata.csv
    ├── Edit_prompt
    ├── videos
    ├── images
    └── bmasks

Metadata Fields

metadata.csv uses relative paths so the dataset can be moved or uploaded without exposing local file-system information.

• —dataset_name: subset identifier.
• —edit_file: source JSON prompt file under Edit_prompt/.
• —id, video_name: sample identifiers.
• —editing_type_id: integer edit category from the prompt file.
• —difficulty_level: difficulty label when available. This field is populated in NR_Benchmark_data_v1 and may be empty in NR_Benchmark_data.
• —file_name: relative path to the source video. This column follows Hugging Face dataset conventions for media files.
• —video_path: same relative video path, kept for readability in custom loaders.
• —image_dir: directory containing extracted source frames.
• —mask_dir: directory containing binary masks for editable regions.
• —save_dir: suggested output directory name used by evaluation scripts.
• —source_prompt, target_prompt, negative_prompt: source caption, target edit caption, and negative prompt.
• —source_object, target_object: source and target object or motion descriptions.
• —instruction: natural-language edit instruction.
• —multiple_choice_question, source_yes_no_question, target_yes_no_question: prompt-level evaluation questions.
• —tweak_index, tstrong_index: indices used by editing/evaluation pipelines.
• —material_type, physics_criteria, deformation_question: non-rigid deformation and physical plausibility annotations.
• —temporal_consistency_question, temporal_flicker_check: temporal quality annotations.
• —instruction_task_type, instruction_comparative_question, instruction_expected_choice: VLM-style instruction following annotations.
• —deformation_config, temporal_qa, instruction_qa: compact JSON copies of the nested annotation objects.

Loading Example

from datasets import load_dataset

# After uploading this folder to Hugging Face:
ds_v1 = load_dataset("<anonymous-or-final-dataset-id>", "nr_benchmark_data_v1", split="benchmark")
ds_dev = load_dataset("<anonymous-or-final-dataset-id>", "nr_benchmark_data", split="benchmark")

Croissant Notes

Hugging Face can generate a Croissant file from the dataset card and tabular metadata. For best results:

1. 1.Upload this README.md, both metadata.csv files, and the referenced videos/, images/, bmasks/, and Edit_prompt/ folders.
2. 2.Keep all paths relative to the dataset root, as written in the CSV files.
3. 3.Do not upload local caches, model checkpoints, evaluation outputs, API keys, or machine-specific config files.
4. 4.After upload, use the Hugging Face dataset viewer or Croissant preview to confirm that both configs are detected.

Intended Use

The benchmark is intended for research evaluation of instruction-driven non-rigid video editing, including edit success, structure preservation, mask-region/background preservation, temporal consistency, visual quality, and physical plausibility.

Limitations

The benchmark is designed for evaluation rather than model training. It may not cover all object categories, motions, camera movements, or physical interactions. Some annotations are phrased for VLM-based automated evaluation and should be checked before adapting them to a different evaluator.

NRVBench License

NRVBench contains resources with mixed licensing.

--Source videos

Source videos are derived from DAVIS and Pexels and remain subject to their original licenses and terms of use. We do not claim ownership of third-party source videos.

NRVBench-created assets

The NRVBench-created annotations, prompts, diagnostic questions, metadata, and masks are released under CC BY-NC 4.0.

--Code

Evaluation scripts and benchmark utilities are released under Apache-2.0 or MIT.

--Provenance

Per-sample source provenance is provided in metadata.csv or source_licenses.csv. Users are responsible for complying with the original licenses of DAVIS and Pexels.