datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
PhysicalAI-WorldModel-Synthetic-Autonomous-Driving-Scenarios
Dataset Description:
PhysicalAI-WorldModel-Synthetic-Autonomous-Driving-Scenarios is a large-scale synthetic video dataset of autonomous-driving scenes generated with NVIDIA's internal Omniverse simulation platform. Each clip is a temporally consistent multi-camera surround capture of one ego vehicle and surrounding traffic participants, paired with per-camera VLM captions. The dataset is designed to fill gaps in real-world driving data along two axes: (1) targeted long-tail… See the full description on the dataset page: https://huggingface.co/datasets/nvidia/PhysicalAI-WorldModel-Synthetic-Autonomous-Driving-Scenarios.minuszero-indian-autonomous-driving-dataset-v2
INDUS-AD: Indian Dataset of Unstructured Urban Scenes for Autonomous Driving
Overview
INDUS-AD is the largest publicly released Indian autonomous-driving dataset for end-to-end autonomous-driving research. Its name expands to Indian Dataset of Unstructured Urban Scenes for Autonomous Driving.
This gated dataset is the decoded companion to the Minus Zero Indian Urban Autonomous Driving Dataset. It provides directly usable camera MP4s, normalized sensor tables… See the full description on the dataset page: https://huggingface.co/datasets/gagandeepreehal/minuszero-indian-autonomous-driving-dataset-v2.cmht-autonomous-driving
Dataset Card for CMHT Autonomous Driving Multimodal (MCAP)
This is a FiftyOne dataset with 4 samples.
Installation
If you haven't already, install FiftyOne:
pip install -U fiftyone
Usage
import fiftyone as fo
from fiftyone.utils.huggingface import load_from_hub
# Load the dataset
# Note: other available arguments include 'max_samples', etc
dataset = load_from_hub("Voxel51/cmht-autonomous-driving")
# Launch the App
session =… See the full description on the dataset page: https://huggingface.co/datasets/Voxel51/cmht-autonomous-driving.MIST-autonomous-driving-dataset
🛣️MIST
Multi-Domain Synthetic Dataset for Rural Driving🌾
🤗 Hugging Face
|
📄 Paper(coming soon)
|
💻 Code(coming soon)
🚗 Simulator (slowroads.io)
📘Dataset Introduction
MIST is a large-scale multi-domain synthetic dataset designed for rural driving scenarios.
It provides explicitly structured domain factors—season, time of day, and weather—forming 32 balanced domain configurations.… See the full description on the dataset page: https://huggingface.co/datasets/jongwonryu/MIST-autonomous-driving-dataset.minuszero-indian-autonomous-driving-dataset
Minus Zero Indian Urban Autonomous Driving Dataset
Overview
This dataset provides original multicamera autonomous-driving recordings in MCAP format. It is designed for non-commercial research on surround-view perception, temporal and cross-camera synchronization, H.265 video pipelines, localization, GNSS/pose integration, and robotics data tooling.
Recordings include camera and GNSS/pose streams, with machine-state telemetry present in a small subset. Camera… See the full description on the dataset page: https://huggingface.co/datasets/gagandeepreehal/minuszero-indian-autonomous-driving-dataset.autonomous-driving-carla
CARLA Autonomous Driving Dataset
Custom datasets for autonomous driving in CARLA simulator
Created for CMPE 789 - Robot Perception at Rochester Institute of Technology
📊 Dataset Overview
This repository contains two custom-generated datasets from the CARLA 0.9.15 simulator for training autonomous driving perception models:
Dataset
Task
Images
Format
Size
YOLO Dataset
Object Detection
4,000
YOLOv8/v11
~1.2 GB
UFLD Dataset
Lane Detection
10,000… See the full description on the dataset page: https://huggingface.co/datasets/jkdxbns/autonomous-driving-carla.v2v-autonomous-driving-qa
V2V & Autonomous Driving QA Dataset
This dataset contains processed research papers focused on V2V communication, autonomous driving, sensor fusion, ADAS, and related fields.It provides structured text sections and automatically generated question–answer (QA) pairs, making it useful for fine-tuning language models, evaluation, or RAG pipelines.
📊 Dataset Summary
~22,457 papers processed
Sections extracted: abstract, introduction, conclusion
55k+ QA pairs… See the full description on the dataset page: https://huggingface.co/datasets/BuRabea/v2v-autonomous-driving-qa.minuszero-indian-autonomous-driving-monocam
Minus Zero Indian Urban Autonomous Driving Dataset - Single Camera
Overview
This dataset provides original single-camera autonomous driving recordings in MCAP format. It is designed for research on camera perception, H.265 video pipelines, localization, GNSS/pose integration, and robotics data tooling.
Depending on the recording, supporting channels include recorded or live GNSS/pose.
The dataset is public for personal, educational, and research use under CC BY-NC… See the full description on the dataset page: https://huggingface.co/datasets/gagandeepreehal/minuszero-indian-autonomous-driving-monocam.minuszero-indian-autonomous-driving-multicam-lidar
Minus Zero Indian Urban Autonomous Driving Dataset - Multicamera LiDAR
Overview
This dataset provides original multicamera autonomous-driving recordings with LiDAR in MCAP format. It is designed for non-commercial research on camera and LiDAR perception, sensor synchronization, H.265 video pipelines, localization, GNSS/pose integration, and robotics data tooling.
The dataset is public for personal, educational, and research use under CC BY-NC 4.0. Commercial use… See the full description on the dataset page: https://huggingface.co/datasets/gagandeepreehal/minuszero-indian-autonomous-driving-multicam-lidar.cmht-autonomous-driving-demo
CMHT Autonomous Driving Multimodal (Demo Subset)
This is a 2-episode preview subset (night_rain_1, night_rain_2) of the
full Voxel51/cmht-autonomous-driving
multimodal MCAP dataset, created specifically so the
interactive FiftyOne Space
fits within a free Hugging Face Space's 50GB disk limit.
For the full 4-episode dataset, complete dataset card, citation, and license
details, see
Voxel51/cmht-autonomous-driving.
Usage
import fiftyone as fo
from… See the full description on the dataset page: https://huggingface.co/datasets/harpreetsahota/cmht-autonomous-driving-demo.CARLA_e2e_autonomous_drivingautonomous-driving-rss-traffic-flow-coherence-state-scoring-v0.1What this dataset tests
Whether a system can score traffic-flow coherence
before and after an ego action.
This is not collision detection.
It measures systemic stability.
Required outputs
pre_action_coherence_score
post_action_coherence_score
coherence_delta
shockwave_generation_flag
braking_propagation_depth
systemic_risk_score
Scoring conventions
coherence scores range 0 to 1
coherence_delta may be negative or positive
shockwave flag is 0 or 1
braking propagation depth… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/autonomous-driving-rss-traffic-flow-coherence-state-scoring-v0.1.autonomous-driving-social-coherence-field-mapping-v0.1What this dataset tests
Whether a system can score
the coherence of a multi-agent intention field.
This is not collision prediction.
It is social alignment measurement.
Required outputs
dominant_scene_intention
coherence_score
tension_index
conflict_pairs
cooperative_clusters
right_of_way_clarity
Scoring conventions
coherence and tension range 0 to 1
right_of_way_clarity is low, medium, or high
conflict_pairs names agent pairs likely to contest the same space… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/autonomous-driving-social-coherence-field-mapping-v0.1.autonomous-driving-intention-field-extraction-v0.1What this dataset tests
Whether a system can infer agent intentions
from context cues in complex driving scenes.
This is not trajectory prediction.
It is intention inference.
Required outputs
agent_id
inferred_intention
intention_confidence
time_horizon_s
alternative_intentions
stability_score
Scoring conventions
confidence and stability range 0 to 1
time horizon is seconds into the near future
Use case
Layer one of Intention Field and Social Coherence Maps.
This enables… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/autonomous-driving-intention-field-extraction-v0.1.minuszero-indian-autonomous-driving-dataset-v2-anonymized
MinusZero Indian Autonomous Driving Dataset V2 Anonymized
This public, manually gated dataset is being prepared. Payload publication is
blocked until privacy, temporal, and immutable remote verification complete.
autonomous-driving-ethical-cost-field-construction-v0.1
What this dataset tests
Whether an intelligence system can constructan ethical cost field for a driving scene.
The task is not to choose an action.The task is to model how harm distributes across agents.
Required outputs
ethical cost field
agent harm vectors
aggregate deformation score
rights infringement index
uncertainty band
Use case
Foundation layer for ethical navigation systems.Trains models to map harm before selecting actions.… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/autonomous-driving-ethical-cost-field-construction-v0.1.autonomous-driving-counterfactual-branch-generation-v0.1What this dataset tests
Whether a system can generate plausible counterfactual branches
from a seed driving scene.
This is not imagination.
It is constrained alternative generation.
Required outputs
branch_id
initiating_agent
counterfactual_action
plausibility_score
simulated_outcome_summary
branch_tree_depth
Scoring conventions
plausibility ranges 0 to 1
depth is the branch depth in the scenario tree
outcome summary describes the simulated end state and recovery
Use case… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/autonomous-driving-counterfactual-branch-generation-v0.1.autonomous-driving-counterfactual-stability-collapse-severity-scoring-v0.1What this dataset tests
Whether a system can score
how severely a plausible counterfactual branch
collapses scene stability.
This is not collision prediction.
It is collapse severity measurement.
Required outputs
coherence_decay_score
cascade_length
recovery_window_s
collapse_severity_index
systemic_fragility_flag
Scoring conventions
scores range 0 to 1
cascade length counts distinct downstream disturbances
recovery window is time available before instability becomes hard to… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/autonomous-driving-counterfactual-stability-collapse-severity-scoring-v0.1.autonomous-driving-driver-state-manifold-estimation-v0.1What this dataset tests
Whether a system can infer driver state
from cabin signals and driving context.
The output is a state manifold vector.
Not a single label.
Required outputs
driver_state_label
fatigue_score
distraction_score
agitation_score
confidence_estimate
state_transition_risk
Scoring conventions
all scores range 0 to 1
labels are baseline, fatigued, distracted, agitated, mixed
transition risk flags likelihood of deterioration in the next window
Use case
Layer one… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/autonomous-driving-driver-state-manifold-estimation-v0.1.autonomous-driving-multisensor-coherence-baseline-modeling-v0.1What this dataset tests
Whether a system can model
the expected coherence of a sensor suite
for a given driving context.
The output is a baseline and tolerance band.
This is the reference for later decoherence detection.
Required outputs
baseline_coherence_score
expected_sensor_alignment
cross_modal_correlation
stability_band
drift_tolerance
baseline_confidence
Scoring conventions
all scores range 0 to 1
stability band is a low-high interval
drift tolerance encodes how much… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/autonomous-driving-multisensor-coherence-baseline-modeling-v0.1.autonomous-driving-ethical-stability-accountability-mapping-v0.1
What this dataset tests
Whether a system can evaluatehow a driving decisionaffects overall scene stabilityand who carries responsibilityfor resulting disturbance.
Required outputs
stability impact description
accountability nodes
stability score
accountability score
recovery quality
Use case
Final layer of ethical navigation stack.
Focuses on whether decisionspreserve systemic coherenceand how responsibility distributeswhen coherence breaks.… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/autonomous-driving-ethical-stability-accountability-mapping-v0.1.autonomous-driving-human-vehicle-coupling-coherence-scoring-v0.1What this dataset tests
Whether a system can score coherence
between driver state, vehicle behavior, and scene context.
This is not crash prediction.
It is coupling integrity.
Required outputs
coupling_coherence_score
overassertive_flag
underassertive_flag
trust_stability_index
takeover_risk_score
recovery_margin
Scoring conventions
all scores range 0 to 1
flags are 0 or 1
takeover risk estimates likelihood of manual override in the next window
Use case
Layer two of… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/autonomous-driving-human-vehicle-coupling-coherence-scoring-v0.1.autonomous-driving-rss-incoherence-propagation-and-shockwave-detection-v0.1What this dataset tests
Whether a system can quantify
how incoherence propagates through traffic.
This is not collision detection.
It is shockwave and recovery measurement.
Required outputs
initial_disturbance_type
affected_agents_count
braking_wave_velocity
lane_stability_loss
recovery_time_s
propagation_severity_score
Scoring conventions
braking_wave_velocity is relative wave speed
lane_stability_loss ranges 0 to 1
propagation severity ranges 0 to 1
affected agents counts… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/autonomous-driving-rss-incoherence-propagation-and-shockwave-detection-v0.1.autonomous-driving-catastrophic-plausible-alternative-identification-v0.1What this dataset tests
Whether a system can identify
the most dangerous coherent alternative
within a counterfactual scenario tree.
Danger is defined as:
high plausibility
high collapse severity
short recovery window.
Required outputs
most_dangerous_branch_id
initiating_agent
trigger_action
time_to_instability_s
prevention_leverage_point
countermeasure_suggestion
Scoring conventions
time_to_instability is seconds
prevention leverage point names the earliest controllable step… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/autonomous-driving-catastrophic-plausible-alternative-identification-v0.1.autonomous-driving-decoherence-onset-detection-v0.1What this dataset tests
Whether a system can detect
the onset of system-wide decoherence.
Decoherence means:
camera, lidar, radar, and map
stop supporting a unified scene narrative.
Required outputs
decoherence_onset_timestamp
coherence_drop_delta
affected_modalities
narrative_conflict_flag
onset_confidence
early_warning_score
Scoring conventions
timestamp is seconds from window start
coherence drop delta is 0 to 1
conflict flag is 1 when the narratives diverge
early warning… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/autonomous-driving-decoherence-onset-detection-v0.1.autonomous-driving-minimal-harm-gradient-pathfinding-v0.1
What this dataset tests
Whether a system can navigatea minimal-harm gradient through a driving scene.
The task is to identify the paththat minimizes total deformationacross all agents.
Required outputs
gradient vectors across actions
minimal harm path
deformation score
stability margin
Use case
Second layer of ethical navigation stack.
Transforms ethical cost fieldinto an actionable path.
Evaluation
Predictions must:
describe gradient… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/autonomous-driving-minimal-harm-gradient-pathfinding-v0.1.autonomous-driving-graceful-degradation-policy-triggering-v0.1What this dataset tests
Whether a system can respond to system-wide decoherence
with a controlled policy degradation.
This is not "stop the car."
It is safe control under uncertainty.
Required outputs
degradation_policy_id
response_mode
speed_adjustment
sensor_reweighting_plan
driver_alert_strategy
recovery_exit_condition
Scoring conventions
speed adjustment is a signed delta (mph)
reweight plan must name which modalities to downweight or upweight
exit condition must be… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/autonomous-driving-graceful-degradation-policy-triggering-v0.1.autonomous-driving-driver-vehicle-coherence-optimal-policy-selection-v0.1What this dataset tests
Whether a system can choose a vehicle policy
that maximizes coherence across:
driver state
vehicle behavior
scene context.
This is not a single driving style.
It is policy manifold navigation.
Required outputs
selected_policy_id
policy_mode
predicted_coherence_trajectory
intervention_intensity
communication_strategy
policy_switch_trigger
Scoring conventions
trajectory is a sequence of coherence values 0 to 1
intensity is low, medium, or high
switch… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/autonomous-driving-driver-vehicle-coherence-optimal-policy-selection-v0.1.autonomous-driving-annotation-standards
Autonomous Driving Scene Annotation Guidelines: LiDAR + Camera Labeling Standards
Overview
Perception is the foundation layer of every autonomous driving stack. Before a planning module can decide whether to brake, merge, or yield, a perception model has to correctly answer a much simpler question: what is out there, and where exactly is it? That answer is only as good as the ground truth data used to train it — which means annotation quality is not a downstream… See the full description on the dataset page: https://huggingface.co/datasets/precisebposolution/autonomous-driving-annotation-standards.autonomous-driving-conflict-likelihood-and-resolution-window-v0.1What this dataset tests
Whether a system can predict conflict likelihood
based on intention divergence and social tension.
This is not collision prediction.
It is conflict risk prediction with an intervention window.
Required outputs
conflict_probability
time_to_conflict_s
resolution_window_s
safest_intervention_type
intervention_latency_budget_s
Scoring conventions
probabilities range 0 to 1
time values are seconds
intervention latency budget must be <= resolution window
Use… See the full description on the dataset page: https://huggingface.co/datasets/ClarusC64/autonomous-driving-conflict-likelihood-and-resolution-window-v0.1.