Datasetpublic

RL-MIND/NJU-HARD-Tracking

NJU-HARD-Tracking Multi-Object Tracking across 122 MP UAV Image Sequences 🤗 Hugging Face · 🟣 ModelScope · 📊 Statistics: HF / MS English | 中文: Hugging Face · ModelScope 🌍 Overview NJU-HARD-Tracking provides the multi-object-tracking release of HARD, with full-resolution frames, temporal ordering, and the original instance annotations. It supports studying how detection and association behave across wide-area aerial… See the full description on the dataset page: https://huggingface.co/datasets/RL-MIND/NJU-HARD-Tracking.

sourceHugging Faceunknownupdated 4d agoView on Hugging Face

0likes681downloads

Dataset Card

NJU-HARD-Tracking

Multi-Object Tracking across 122 MP UAV Image Sequences

<a href="https://huggingface.co/datasets/RL-MIND/NJU-HARD-Tracking">🤗 Hugging Face</a> · <a href="https://www.modelscope.cn/datasets/KAIWANG/NJU-HARD-Tracking">🟣 ModelScope</a> · 📊 Statistics: <a href="https://huggingface.co/datasets/RL-MIND/NJU-HARD-Tracking/blob/main/statistics.json">HF</a> / <a href="https://www.modelscope.cn/datasets/KAIWANG/NJU-HARD-Tracking/file/view/master/statistics.json?status=1">MS</a>

English | 中文: <a href="https://huggingface.co/datasets/RL-MIND/NJU-HARD-Tracking/blob/main/READMEZH.md">Hugging Face</a> · <a href="https://www.modelscope.cn/datasets/KAIWANG/NJU-HARD-Tracking/file/view/master/READMEZH.md?status=1">ModelScope</a>

🌍 Overview

NJU-HARD-Tracking provides the multi-object-tracking release of HARD, with full-resolution frames, temporal ordering, and the original instance annotations. It supports studying how detection and association behave across wide-area aerial scenes.

The source contains 3,549 original JPEG frames, 21 sequences, and 138,830 box annotations across four classes: pedestrian, car, bus, and truck. Every frame is 12,768 × 9,564 pixels (approximately 122 MP). The imagery covers sparse and dense scenes, with substantial variation in target scale and camera motion.

HARD stands for Ultra-High-resolution Airborne Remote-sensing Dataset. The NJU-HARD-Detection release contains the same frame and annotation records with a task-specific card. Both releases preserve the original JPEG bytes and labels. Their repository names identify these task releases; they do not denote separate acquisitions.

✨ What the Data Covers

—Wide-area detail: complete 122 MP frames retain small pedestrian and vehicle targets within a broad field of view.
—Temporal structure: 21 ordered sequences connect per-frame localization with cross-frame association.
—Diverse viewing conditions: the source manuscript describes 50–355 m acquisition heights and four flight modes: linear-nadir, linear-oblique, stationary-gaze, and orbit-gaze.
—Realistic imbalance: cars dominate the annotations, while buses and trucks are less frequent. Scene density ranges from approximately 12.4 to 79.8 annotated objects per frame across the three capture subsets.

These properties support research on wide-area spatio-temporal scene understanding. They do not establish a performance claim for any model.

📊 Dataset at a Glance

Item	Source inventory represented by the release
Original frames	3,549
Image resolution	12,768 × 9,564 pixels
Capture subsets / sequences	3 / 21
Box annotation rows	138,830
Frames with a source label file	3,544
Frames without a source label file	5; annotation state is missing
Source JPEG payload	368,182,969,599 bytes, approximately 368.18 GB
Storage unit	One original frame per Parquet row
Configuration	`default`
Splits	`train`, `validation`, `test`
Separate browsing preview	`preview` / `sample`: 7 thumbnail frames in separate Parquet files, at most 1,024 pixels on the longest edge

The JPEG byte count describes the original image payload, not the final Parquet file size. In default, each image embeds its original bytes and has path=null; it is not resized, cropped, or re-encoded. The separate seven-frame preview uses derived thumbnails and is not included in the 3,549-frame total. It spans the three subsets and splits, including examples with missing labels, repeated identities, and out-of-bounds boxes. Box totals count annotation rows across frames, not distinct physical objects or verified unique trajectories.

Dense `category`	Class	Original `source_class_id`	Annotation rows
0	pedestrian	0	27,711
1	car	2	103,361
2	bus	5	4,033
3	truck	7	3,725

The four source-ID totals match the manuscript's per-class totals exactly. category provides a contiguous label space; source_class_id preserves the original 0/2/5/7 values.

🗂️ Sequence Splits

The releases use the sequence-level split in the manuscript's Table 4. All frames from a sequence stay together. The mapping from the manuscript's A/B/C subset codes to source directories was reconstructed from sequence inventories and verified by exact agreement with all three split annotation totals.

Split	Sequences	Frames	Annotation rows
`train`	13	2,055	89,148
`validation`	3	551	15,820
`test`	5	943	33,862

Source directory / paper code	`train` sequence IDs	`validation`	`test`
`UAV-BASE-13` / A	1, 2, 3, 4, 5	6	—
`UAV-BASE-14` / B	1, 2, 5, 8, 9	3, 7	4, 6
`NJU-xianlin` / C	1, 5, 6	—	2, 3, 4

The split is sequence-disjoint, not site-disjoint. The manuscript's tracking evaluation pools the five test sequences with TrackEval's combined convention. The complete tracking release includes all 21 sequences for training, validation, and testing.

🧩 Record Format

Each Parquet row is one frame. The detection and tracking releases share this schema so that image bytes, labels, and source provenance remain comparable.

Field	Meaning
`image`, `preview_only`	Native image value with a null path; original JPEG in `default`, a derived thumbnail when `preview_only=true`
`image_id`, `scene`, `sequence_id`	Frame identity, capture subset, and sequence identity
`source_sequence_id`, `paper_sequence_id`	Original sequence number and corresponding A/B/C manuscript sequence code
`frame_id`, `frame_name`	Integer parsed from the original frame filename, and that filename; the original starting index is retained
`split`, `width`, `height`	Assigned source-sequence split and dimensions of the current `image.bytes`
`source_width`, `source_height`	Original full-resolution dimensions, also retained in the preview
`source_image_path`, `source_image_sha256`	Relative source path and original JPEG checksum
`image_sha256`	Checksum of the current `image.bytes`; identical to the source checksum in `default`, different for a derived preview
`annotation_status`, `num_objects`	`present` or `missing`; object count is null when the source label is missing
`source_annotation_path`, `raw_annotation_bytes`, `annotation_sha256`	Original label provenance and exact bytes; bytes/checksum are null for missing labels
`quality_flags`	Frame-level source-quality indicators
`objects`	List of object records, or null for a missing source label

Each object includes:

Field	Meaning
`id`, `source_line`	Source label path plus row identity, and the original line number
`category`, `source_class_id`	Contiguous class label and unchanged source class ID
`source_track_id`, `source_track_key`	Raw instance ID and its sequence/class-scoped key
`source_bbox_normalized`	Original YOLO-style `[cx, cy, width, height]` in normalized coordinates
`bbox`, `area`	COCO-style `[x_min, y_min, width, height]` and area in the current image's pixels, retained as floating point
`quality_flags`	Object-level source-quality indicators

For tracking, process frames within each sequence_id in increasing frame_id order. source_track_key includes the sequence, source class, and raw instance ID; it prevents cross-class ID collisions but does not resolve the same-class ambiguities listed below. Preserve source_track_id when exporting or comparing source labels.

Pixel boxes are derived from the original normalized center/size coordinates and the current width/height. They use original-image pixels in default and thumbnail pixels in preview; they are not clipped or rounded to integers. No actual capture timestamp or measured video frame rate is inferred from the frame index. The manuscript's P = 2 s is recorded as evaluation context, not synthesized per-frame timing metadata.

⚠️ Source Annotation Quality

The release preserves the audited source rather than silently changing ambiguous annotations. Quality flags and original label bytes make the following issues visible:

Source issue	Count / interpretation
Missing label files	5 frames; `objects=null`, not an empty list or a confirmed negative frame
Raw IDs reused across classes within a sequence	158 sequence/ID groups; do not use raw ID alone as a track key
Same class and ID repeated within a frame	24 groups; a class-scoped key still has source ambiguity in these cases
Exact duplicate annotation row	1 duplicate occurrence, retained
Box corners beyond normalized image bounds	2,149 rows at a tolerance of `1e-6`; original coordinates retained

The missing label files correspond to UAV-BASE-13 sequence 1 frame 0, sequence 2 frame 0, and sequence 6 frames 0–2. There are no empty source TXT files. Downstream experiments should state how they treat missing labels, duplicate identities, duplicate rows, and out-of-bounds boxes. A cleaned evaluation protocol must document its transformations separately; the source track keys are not a claim of fully resolved ground-truth identities.

🚀 Quick Start

Start with the small preview to inspect the schema and quality flags. With Hugging Face datasets, streaming and disabled image decoding keep this inspection lightweight:

python

from datasets import Image, load_dataset

data = load_dataset(
    "RL-MIND/NJU-HARD-Tracking",
    name="preview",
    split="sample",
    streaming=True,
).cast_column("image", Image(decode=False))

frame = next(iter(data))
print(frame["sequence_id"], frame["frame_id"], frame["annotation_status"])
if frame["objects"] is None:
    print("Missing source label; this is not a negative frame.")
else:
    for obj in frame["objects"]:
        print(obj["source_track_key"], obj["bbox"], obj["quality_flags"])

ModelScope's dataset loader exposes the same preview:

python

from modelscope.msdatasets import MsDataset

data = MsDataset.load(
    "KAIWANG/NJU-HARD-Tracking",
    subset_name="preview",
    split="sample",
    use_streaming=True,
)
frame = next(iter(data))
print(frame["sequence_id"], frame["frame_id"], frame["annotation_status"])

For original frames, use name="default", split="test" on Hugging Face, or subset_name="default", split="test" on ModelScope; train and validation are also available. The preview is a browsing aid, not a continuous tracking evaluation sequence. Use default for full-resolution experiments and chronological tracking. Full-resolution frames remain large even when streamed; after resizing an image, transform its boxes consistently.

📄 License and Research Context

License metadata is unknown. The source manuscript describes research-only distribution and prohibits attempts to re-identify individuals or vehicles, but a complete license agreement is not supplied with the source material reviewed for this release.

The detection task uses per-frame box supervision; the tracking task additionally studies sequence-local association. The manuscript reports COCO detection metrics and HOTA/s-HOTA tracking evaluation. This data package does not, by itself, establish a validated implementation of the manuscript's latency-accounting protocol.