CoolFace
Datasetpublic

AnTo2209/MarineEVT

MarineEVT Dataset MarineEVT: Advancing Event-Centric Marine Video Understanding via Visual Tool Reasoning 📖 Description MarineEVT is a comprehensive event-centric dataset and benchmark for marine video understanding. It comprises 20,000 richly annotated underwater video question-answer pairs spanning 20 fine-grained dimensions, designed to support semantic, contextualized, spatial-temporal, and causal reasoning in marine environments. The dataset addresses the… See the full description on the dataset page: https://huggingface.co/datasets/AnTo2209/MarineEVT.

sourceHugging Faceupdated 3mo agoView on Hugging Face
0likes1.4kdownloads
Dataset Card

MarineEVT Dataset

![Paper](https://marineevt.hkustvgd.com/) ![Dataset Size]() ![License]()

MarineEVT: Advancing Event-Centric Marine Video Understanding via Visual Tool Reasoning

📖 Description

MarineEVT is a comprehensive event-centric dataset and benchmark for marine video understanding. It comprises 20,000 richly annotated underwater video question-answer pairs spanning 20 fine-grained dimensions, designed to support semantic, contextualized, spatial-temporal, and causal reasoning in marine environments.

The dataset addresses the challenge that informative events in marine videos are often sparse, ephemeral, and unevenly distributed, posing significant challenges for existing Video Language Models (VLMs).

📊 Dataset Statistics

MetricValue
Total QA Pairs20,000
Evaluation Set2,000 pairs (reserved for testing)
Dimensions20 fine-grained categories
Video SourcesUnderwater marine environments

Reasoning Types

The dataset covers five major reasoning categories:

  1. 1.Semantic Reasoning - Understanding what events happened and their semantic meaning
  2. 2.Contextual Reasoning - Identifying which entities are present in video events
  3. 3.Spatial Reasoning - Localizing where entities are in the underwater scene
  4. 4.Temporal Reasoning - Understanding when events occur and temporal dynamics
  5. 5.Causal Reasoning - Inferring causal relationships and why events occur

📁 Dataset Structure

MarineEVT/
├── CasualReasoning/
│   ├── Human-SpeciesCasualDynamics/
│   │   ├── train/
│   │   │   ├── videos/
│   │   │   └── multi_turn_data_ver2.json
│   │   └── test/
│   │       ├── videos/
│   │       └── multi_turn_data_ver2.json
│   ├── Inter-SpeciesCausalDynamics/
│   └── ReasonInference/
├── SpatialReasoning/
│   ├── [subdimension]/
│   │   ├── train/
│   │   └── test/
├── SemanticReasoning/
├── ContextualReasoning/
├── TemporalReasoning/
└── README.md

Data Format

Each JSON file contains multi-turn QA pairs with the following structure:

json
{
  "video_id": "zoDLceQg0J2U",
  "video_url": "",
  "question": "What is the marine animal doing?",
  "answer": "The animal is hunting for prey...",
  "question_task": "VideoQuestionAnswering",
  "dimension": "CasualReasoning",
  "subdimension": "Human-SpeciesCausalDynamics",
  "turns": [
    {
      "turn_id": 1,
      "id": 1,
      "visual_input": [...],
      "user_query": "...",
      "assistant_response": "..."
    }
  ]
}

📥 Download

The dataset is available for download at:

Official Website: https://marineevt.hkustvgd.com/

Hugging Face: Use this dataset directly via the datasets library:

python
from datasets import load_dataset

# Load the dataset
dataset = load_dataset("your-username/marineevt")

# Access training and test splits
train_data = dataset["train"]
test_data = dataset["test"]

🔧 Usage Example

python
import json
from pathlib import Path

# Load training data
with open("CasualReasoning/Human-SpeciesCasualDynamics/train/multi_turn_data_ver2.json", "r") as f:
    data = json.load(f)

# Access a sample
sample = data[0]
print(f"Question: {sample['question']}")
print(f"Answer: {sample['answer']}")
print(f"Dimension: {sample['dimension']}")

📄 License

This dataset is released under the MIT License.

📝 Citation

If you use this dataset in your research, please cite our paper:

bibtex
@inproceedings{to2026marineevt,
  title={{MarineEVT}: Advancing Event-Centric Marine Video Understanding via Visual Tool Reasoning},
  author={To, Tuan-An and Wong, Yuk-Kwan and Vu, Tuan-Anh and Zheng, Ziqiang and Yeung, Sai-Kit},
  booktitle={European Conference on Computer Vision (ECCV)},
  year={2026}
}

🔗 Links