Datasetpublic

RL-MIND/NJU-HARD

NJU-HARD 🤗 Hugging Face · 🟣 ModelScope · 📊 Statistics English | 中文：Hugging Face · ModelScope 📚 Introduction NJU-HARD is the deduplicated, full-resolution release of the HARD visual question answering data. It contains 1,563 valid VQA records across 8 task types, using 854 original aerial images. Only images referenced by these valid questions are included. Every original JPEG is embedded once in a native… See the full description on the dataset page: https://huggingface.co/datasets/RL-MIND/NJU-HARD.

sourceHugging Faceunknownupdated 9d agoView on Hugging Face

0likes570downloads

Dataset Card

NJU-HARD

<p align="center"> <a href="https://huggingface.co/datasets/RL-MIND/NJU-HARD">🤗 Hugging Face</a> · <a href="https://www.modelscope.cn/datasets/KAIWANG/NJU-HARD">🟣 ModelScope</a> · <a href="https://huggingface.co/datasets/RL-MIND/NJU-HARD/blob/main/statistics.json">📊 Statistics</a> </p>

<p align="center"><b>English</b> | 中文：<a href="https://huggingface.co/datasets/RL-MIND/NJU-HARD/blob/main/READMEZH.md">Hugging Face</a> · <a href="https://www.modelscope.cn/datasets/KAIWANG/NJU-HARD/file/view/master/READMEZH.md?status=1">ModelScope</a></p>

📚 Introduction

NJU-HARD is the deduplicated, full-resolution release of the HARD visual question answering data. It contains 1,563 valid VQA records across 8 task types, using 854 original aerial images. Only images referenced by these valid questions are included.

Every original JPEG is embedded once in a native Hugging Face Parquet image table. Questions refer to images by their full SHA-256 IDs, in the original Image 1 / Image 2 / Image 3 order. The release supports single-image and multi-image VQA without storing the same original repeatedly for different questions.

This release packages the question table and original-image table together for distribution on Hugging Face and ModelScope. It is separate from the per-question embedded-image HARD-VQA version.

📊 Dataset at a Glance

Item	Value
Valid questions	1,563
Unique original images	854 distinct paths and SHA-256 hashes
Ordered image references	2,571
Images per question	1, 2, or 3
Original resolution	12,768 × 9,564 pixels
Original JPEG bytes	87,637,348,481 bytes, approximately 87.64 GB
Original-image Parquet shards	180
All Parquet files	87,639,068,405 bytes, including questions and thumbnail preview
Split	`unsplit`; no new train/validation/test split is introduced
Preview	8 examples, one per task, with derived thumbnails

Original images are not resized, cropped, or re-encoded. Thumbnail images are separate browsing aids and are not additional original training or evaluation images.

Source task label	Questions
`classification`	250
`counting`	200
`property`	154
`spatial_relation`	100
`task1_specialdetect`	66
`task3_region_single`	151
`task4_findit_nearbox`	342
`task_2_relocate`	300
Total	1,563

📦 Data Organization

Configuration	Split	Contents
`questions` (default)	`unsplit`	Question text, options, answers, ordered `image_ids`, and source annotations
`images`	`unsplit`	One original image per row, keyed by `image_id`; JPEG bytes embedded as `Image()`
`preview`	`sample`	Eight VQA examples with thumbnails for browsing

text

questions/   # 1,563 question records
images/      # 854 originals in 180 Parquet shards
preview/     # 8 thumbnail examples
provenance/  # source/checksum manifests, image index, and validation reports
statistics.json

The question table preserves question_id, question, options, answer, answer_text, task_type, num_images, source_image_paths, reference_bbox, scene, sequence, and quality_flags. The options struct has A/B/C/D fields; C/D are null for binary-choice questions. Ordered image_ids correspond one-to-one with source_image_paths.

The image table contains image_id, image, source_image_paths, sha256, width, height, and byte_count. image_id equals the SHA-256 of the complete original JPEG. Source paths are provenance only: loading does not require the source server or an external image directory.

🚀 Quick Start

Load the small question table first, or inspect the thumbnail preview:

python

from datasets import load_dataset

REPO = "RL-MIND/NJU-HARD"
questions = load_dataset(REPO, "questions", split="unsplit")
preview = load_dataset(REPO, "preview", split="sample")
print(questions[0]["question"], questions[0]["image_ids"])

To reconstruct a VQA example with original images:

python

from datasets import Image, load_dataset

images = load_dataset(REPO, "images", split="unsplit")
images = images.cast_column("image", Image(decode=False))

# Store only IDs and row numbers in the lookup, not all image bytes.
id_to_row = {image_id: i for i, image_id in enumerate(images["image_id"])}
question = questions[0]
for image_number, image_id in enumerate(question["image_ids"], start=1):
    jpeg_bytes = images[id_to_row[image_id]]["image"]["bytes"]
    # Process this original; iteration preserves the question's image order.
    print(image_number, image_id, len(jpeg_bytes))

Loading questions alone does not download the originals. A full images load downloads approximately 87 GB and builds a local cache; allow additional cache space. Datasets 3.6.0 and 5.0.0 have been checked with this format.

After downloading a complete repository snapshot from either platform, local Parquet loading is also supported:

python

from pathlib import Path
from datasets import load_dataset

root = Path("/path/to/NJU-HARD")
questions = load_dataset("parquet", data_files={
    "unsplit": str(root / "questions" / "*.parquet")
}, split="unsplit")
images = load_dataset("parquet", data_files={
    "unsplit": str(root / "images" / "*.parquet")
}, split="unsplit")

Sequential streaming avoids downloading the entire image table up front:

python

from datasets import Image, load_dataset

stream = load_dataset(REPO, "images", split="unsplit", streaming=True)
stream = stream.cast_column("image", Image(decode=False))
first = next(iter(stream))
print(first["image_id"], len(first["image"]["bytes"]))

Streaming is sequential and does not provide random lookup by image ID. For repeated question-to-image joins, use the cached dataset and row-number index above.

🔍 Validation and Annotation Notes

—The source has 1,565 records. IDs 476 and 496 have null questions, options, and answers and are excluded. Images used only by excluded records are not included.
—ID 597 is preserved with the flag duplicate_option_text: A and B both contain white, while the source answer is B.
—Question text, options, answers, ordered image references, and bounding boxes preserve source values. Structural validation does not establish that every answer is semantically correct.
—Every original JPEG is checked by SHA-256 before embedding and after Parquet readback. Independent validation checks all question fields, image metadata, reference closure, and image-index rows, plus sampled original bytes through both Datasets versions. Reports are in provenance/.
—Preview images have a 1,024-pixel longest edge and preview_only=true. The ordered images list and image_1/image_2/image_3 columns support browsing; missing slots are null. Preview questions and boxes still use original-image coordinates. Use the originals for evaluation.
—The source provides no train/validation/test assignment, so all records remain unsplit. If you create a split, account for shared images and adjacent frames across partitions.

📜 License and Provenance

The source data does not include an explicit license. The dataset card therefore retains `license: unknown`. Packaging the data does not grant a new data-use license; consult the data owner for applicable terms.

Source-image checksums, excluded records, the image-to-shard index, and validation evidence are provided in provenance/. The paired header uses the supplied RL-MIND group mark and a newly generated NJU-HARD dataset logo.