Datasetpublic

nablasinc/Japanese_Video-QA

Japanese Video-QA Overview Japanese Video-QA is a video question-answering benchmark focused on Japanese cultural content, designed to evaluate multimodal large language models (MLLMs) on Japanese-specific videos. 428 YouTube videos → 800 QA pairs 6 domains, 100 sub-domains covering Japanese culture Questions generated by Gemini 2.5 Flash and manually verified Evaluated with LLM-as-a-Judge (GPT-4o), scoring 1–3 Authors: 峯悠大, 新立拓也, 髙橋和也 (NABLAS Inc.)… See the full description on the dataset page: https://huggingface.co/datasets/nablasinc/Japanese_Video-QA.

sourceHugging Facecc-by-4.0updated 7mo agoView on Hugging Face

0likes23downloads

Dataset Card

Japanese Video-QA

Overview

Japanese Video-QA is a video question-answering benchmark focused on Japanese cultural content, designed to evaluate multimodal large language models (MLLMs) on Japanese-specific videos.

—428 YouTube videos → 800 QA pairs
—6 domains, 100 sub-domains covering Japanese culture
—Questions generated by Gemini 2.5 Flash and manually verified
—Evaluated with LLM-as-a-Judge (GPT-4o), scoring 1–3

Authors: 峯悠大, 新立拓也, 髙橋和也 (NABLAS Inc.)

Dataset Fields

Field	Type	Description
`video_id`	string	YouTube video ID
`video_title`	string	Title of the video
`domain`	string	Domain: 四季・行事 / 観光名所 / 伝統文化 / 食文化 / 自然・風景 / ポップカルチャー
`subdomain`	string	Sub-domain (100 categories)
`duration`	string	`short` (< 4 min) or `medium` (4–20 min)
`question`	string	Question in Japanese
`answer`	string	Ground-truth answer (choice label for `multi_choice`)
`type`	string	`open` / `multi_choice` / `yes_no`
`category`	string	`spatial` / `count` / `action` / `temporal` / `causal`
`choices`	list[string]	Answer options (only for `multi_choice`)
`start_time`	string	Segment start time (reference only)
`end_time`	string	Segment end time (reference only)

start_time and end_time are approximate annotations for reference. Models should take the full video as input.

Benchmark Results

Scoring: 1 = incorrect, 2 = partial, 3 = correct

LLM judge: GPT-4o (sample code)

Model	Avg	Score 1	Score 2	Score 3
Gemini 3 Pro	2.61	122	68	610
Gemini 2.5 Flash	2.57	139	68	593
Qwen3-VL-8B-Instruct	2.24	260	89	451
Qwen3-VL-8B-Thinking	2.20	285	70	445
Qwen3-VL-4B-Instruct	2.19	287	77	436
Qwen3-VL-4B-Thinking	2.13	308	77	415
Phi-4-multimodal-instruct	1.74	465	76	259

Usage

python

from datasets import load_dataset

dataset = load_dataset("nablasinc/Japanese_Video-QA")
test = dataset["test"]

Use video_id to retrieve the YouTube video and provide the full video as model input.

License and Copyright

The annotations and metadata in this dataset are released under CC BY 4.0.

The videos referenced in this dataset are YouTube videos, and all copyrights are retained by the respective video uploaders. No video content is redistributed — only YouTube video IDs and metadata are provided. This dataset is intended for research purposes only.

Citation

bibtex

@dataset{japanese_video_qa_2026,
  title   = {Japanese Video-QA: A Benchmark for Evaluating Video Understanding of Japanese Culture},
  author  = {Mine, Yudai and Shintate, Takuya and Takahashi, Kazuya},
  year    = {2026},
  url     = {https://huggingface.co/datasets/nablasinc/Japanese_Video-QA}
}