26TCA/TCA-Bench

TCA-Bench

Official ECCV 2026 benchmark release for the paper "Temporal and Cross-modal Alignment for Enhanced Audiovisual Video Captioning."

TCA-Bench is a diagnostic benchmark for audiovisual video captioning. It evaluates base audio/visual perception, audio-visual binding, and cross-modal temporal reasoning using structured ground truth annotations.

The benchmark contains 459 anonymized short videos. All annotation files use the anonymized mp4 filename as id, matching files in videos/.

Files

• —videos.tar.gz: compressed archive containing anonymized video files named tca_bench_000001.mp4 through tca_bench_000459.mp4.
• —gt/captions.json: Stage-1 ground-truth captions.
• —gt/stage-2.json: Stage-2 audio-visual binding lists.
• —gt/stage-3.json: Stage-3 temporal relation lists.
• —scripts/evaluate.py: evaluation script.
• —scripts/prompts/: captioning and judge prompts.
• —requirements.txt: evaluator dependencies.
• —source_manifest.csv: source URL and temporal segment for each anonymized video.

Evaluation

Stage 1 evaluates base perception against gt/captions.json:

• —1v: visual quality.
• —1a: audio quality.

Stage 2 evaluates audio-visual binding against gt/stage-2.json.

Stage 3 evaluates temporal reasoning against gt/stage-3.json and reports Temporal F1 with precision and coverage.

scripts/prompts/caption_prompt.txt is the recommended prompt for generating candidate captions.

Extract videos.tar.gz in the repository root before reading video files:

tar -xzf videos.tar.gz

Candidate captions should be a JSON array or JSONL file:

{"id": "tca_bench_000001.mp4", "caption": "Candidate caption text..."}

Run evaluation:

python -m pip install -r requirements.txt
export TCA_EVAL_API_KEY="..."
export TCA_EVAL_BASE_URL="https://openrouter.ai/api/v1"
export TCA_EVAL_MODEL="gpt-4.1"
python scripts/evaluate.py --input path/to/captions.json --name my-model --stage 1v 1a 2 3

Outputs are written to results/<name>/<timestamp>/.

License

This release is for non-commercial research use under CC BY-NC-SA 4.0.

Citation

@misc{tca_bench_2026,
  title = {Temporal and Cross-Modal Alignment for Enhanced Audiovisual Video Captioning},
  year = {2026}
}