Datasetpublic

TESS-Computer/carla-simlingo-raw

SimLingo CARLA Dataset (Raw, 4Hz) Raw driving data from CARLA simulator. No transformations or derived fields - all original measurements preserved as-is. Dataset Summary Source: SimLingo (CVPR 2025) Scale: 228,757 frames (23 shards) Frame Rate: 4 FPS Resolution: 1024x512 RGB Routes: Complete driving episodes (routes never split across shards) Column Schema Core Fields Column Type Description route_id string Route… See the full description on the dataset page: https://huggingface.co/datasets/TESS-Computer/carla-simlingo-raw.

sourceHugging Facemitupdated 8mo agoView on Hugging Face

0likes30downloads

Dataset Card

SimLingo CARLA Dataset (Raw, 4Hz)

Raw driving data from CARLA simulator. No transformations or derived fields - all original measurements preserved as-is.

Dataset Summary

—Source: SimLingo (CVPR 2025)
—Scale: 228,757 frames (23 shards)
—Frame Rate: 4 FPS
—Resolution: 1024x512 RGB
—Routes: Complete driving episodes (routes never split across shards)

Column Schema

Core Fields

Column	Type	Description
`route_id`	string	Route identifier
`frame_idx`	int32	Frame index within route
`image`	bytes	Original JPEG image bytes

Control Signals (Raw)

Column	Type	Description
`steer`	float32	Steering [-1, 1]
`throttle`	float32	Throttle [0, 1]
`brake`	bool	Brake applied

Vehicle State

Column	Type	Description
`speed`	float32	Current speed (m/s)
`target_speed`	float32	Target speed
`speed_limit`	float32	Speed limit
`theta`	float32	Heading angle
`angle`	float32	Angle to target

Navigation

Column	Type	Description
`command`	int32	Navigation command
`next_command`	int32	Next navigation command
`pos_global`	string (JSON)	Global position [x, y]
`target_point`	string (JSON)	Target point
`target_point_next`	string (JSON)	Next target point
`aim_wp`	string (JSON)	Aim waypoint
`route`	string (JSON)	Planned route waypoints
`route_original`	string (JSON)	Original route waypoints
`changed_route`	bool	Route was changed

Hazards & Environment

Column	Type	Description
`junction`	bool	In junction
`vehicle_hazard`	bool	Vehicle hazard detected
`vehicle_affecting_id`	int32	ID of affecting vehicle
`walker_hazard`	bool	Pedestrian hazard
`walker_affecting_id`	int32	ID of affecting pedestrian
`light_hazard`	bool	Traffic light hazard
`stop_sign_hazard`	bool	Stop sign hazard
`stop_sign_close`	bool	Stop sign nearby
`walker_close`	bool	Pedestrian nearby
`walker_close_id`	int32	ID of nearby pedestrian
`speed_reduced_by_obj_type`	string	Object type causing speed reduction
`speed_reduced_by_obj_id`	int32	Object ID causing speed reduction
`speed_reduced_by_obj_distance`	float32	Distance to speed-reducing object
`control_brake`	bool	Control brake applied

Augmentation (from SimLingo)

Column	Type	Description
`augmentation_translation`	float32	Translation augmentation
`augmentation_rotation`	float32	Rotation augmentation

Transforms

Column	Type	Description
`ego_matrix`	string (JSON)	4x4 ego vehicle transform matrix
`boxes`	string (JSON)	3D bounding boxes for all objects

Commentary (Optional)

Column	Type	Description
`commentary`	string	Natural language commentary
`commentary_data`	string (JSON)	Full commentary object with metadata

Usage

python

from datasets import load_dataset
import json

ds = load_dataset("TESS-Computer/carla-simlingo-raw", split="train")

sample = ds[0]
print(sample['route_id'])
print(sample['steer'], sample['throttle'], sample['brake'])
print(sample['speed'])

# Parse JSON fields
pos = json.loads(sample['pos_global'])
boxes = json.loads(sample['boxes']) if sample['boxes'] else []

Data Collection

—Simulator: CARLA 0.9.15 (Leaderboard 2.0)
—Expert: PDM-Lite (rule-based, 100% route completion)
—Scenarios: Single-scenario routes with random weather
—Towns: Towns 1-13

Citation

bibtex

@inproceedings{renz2025simlingo,
  title={SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignment},
  author={Renz, Katrin and Chen, Long and Arani, Elahe and Sinavski, Oleg},
  booktitle={CVPR},
  year={2025},
}

License

MIT (dataset processing code). Original data subject to SimLingo and CARLA licenses.