Eventual-Inc/physical-ai-evals-libero-spatial-pilot

LIBERO-Spatial paired pilot

Historical rollout records for OpenVLA and VLA-JEPA on 100 paired LIBERO-Spatial episode specifications. The dataset contains 200 policy/episode records and 23,283 transition rows.

This is an exploratory trace, not a LIBERO reference evaluation or a confirmatory model comparison. The artifact label 2026-07-02 is not a recorded execution timestamp.

Files

• —steps.parquet: normalized rollout-v1 transition rows.
• —episodes.parquet: one row per policy and episode specification.
• —failure_signatures.csv: post-hoc candidate signals for failed episodes.
• —summary.json: descriptive aggregate counts.
• —manifest.json: recorded configuration and provenance limitations.
• —build_bundle.py: deterministic validation and bundle construction.
• —ARTIFACT_README.md: the original artifact documentation.
• —SOURCE_SHA256SUMS: checksums recorded in source control.
• —SHA256SUMS: checksums for the files published here.

Query with Daft

import daft

steps = daft.read_parquet("hf://datasets/Eventual-Inc/physical-ai-evals-libero-spatial-pilot/steps.parquet")
failures = steps.where(steps["success"] == False)
failures.select("policy_type", "episode_id", "task_id", "step_idx").show()

episodes = daft.read_parquet("hf://datasets/Eventual-Inc/physical-ai-evals-libero-spatial-pilot/episodes.parquet")
episodes.select("policy_type", "task_id", "success", "num_steps").show()

Provenance limits

• —The traces store requested seed 7, but the simulator seed is 0/unverified.
• —Policy random-number generator state was not recorded.
• —The OpenVLA checkpoint identity is inferred rather than established by the stored model field.
• —Model revisions, exact evaluation code revision, dependency lock, hardware, container image, and execution timestamps were not recorded.
• —The trace uses 10 fixed initial states per task rather than the 50-trial reference protocol.
• —Referenced frame and video files are not included.
• —Failure signatures are unvalidated candidate labels, not causal diagnoses.

The source artifact was extracted from `c1e39d7`.