HashtagRobotics/tic-tac-toe-so101-block-a-clean-v1

SO-101 Tic-Tac-Toe Block A — Games 1–15

<p align="center"> <a href="https://github.com/Hashtag-Robotics/so101-tic-tac-toe"><img alt="GitHub implementation" src="https://img.shields.io/badge/GitHub-End--to--endimplementation-181717?logo=github"></a> <a href="https://hashtagrobotics.tr/so101-tic-tac-toe"><img alt="Project story" src="https://img.shields.io/badge/Project-Readablestory-ff365d"></a> <a href="https://huggingface.co/HashtagRobotics/smolvla-tic-tac-toe-games-1-15-120k"><img alt="Trained model" src="https://img.shields.io/badge/Model-SmolVLA_120K-7c3aed?logo=huggingface"></a> <img alt="LeRobotDataset v3" src="https://img.shields.io/badge/LeRobotDataset-v3.0-21b8a6"> </p>

A revision-pinned, dual-camera physical-manipulation dataset for language-conditioned placement of 3D-printed tic-tac-toe pieces with an SO-101 follower arm.

Abstract

This dataset contains 195 physical SO-101 demonstrations, 144,723 synchronized frames, and 18 language-conditioned placement tasks. A human operator controlled an SO-101 follower through a leader arm while two RGB cameras observed the tabletop scene and the end-effector workspace. The release covers red X and white O placement into every cell of a 3 × 3 board, organized as 15 game groups with 13 demonstrations per group.

The artifact is serialized as LeRobotDataset v3.0. Every frame provides a six-dimensional follower state, a six-dimensional target action, a top-camera observation, a wrist-camera observation, timing/index metadata, and a task index. This card treats the dataset as a scientific artifact: all quantitative claims below refer to the pinned revision shown in the artifact identity table.

This is a training release, not a benchmark result. It has no official held-out test split and does not, by itself, establish policy success on physical hardware.

Documentation map

Artifact identity

<p align="center"> <img src="./assets/dataset-profile.png" alt="Dataset profile with task coverage and duration distribution" width="96%"> </p>

Experimental system

Physical scene

The manipulation scene consists of:

• —one 3D-printed tic-tac-toe board;
• —red X and white O pieces;
• —a black-framed X staging region and a red-framed O staging region;
• —an SO-101 follower arm;
• —a fixed top camera for global board state; and
• —a wrist camera for near-field grasp and placement geometry.

The printable scene geometry, camera-holder files, simulation configuration, agent code, and end-to-end operating instructions are maintained in the GitHub repository. The physical setup is deliberately task-specific: changes in camera pose, board pose, lighting, print tolerances, gripper geometry, calibration, or servo response can create material distribution shift.

Demonstration unit

Each episode is one commanded pick-and-place trajectory. The command specifies a piece and a board cell, for example:

put the red X in the bottom left cell

At 30 Hz, the recorder stores the follower state, the leader-derived target action, two synchronized image observations, and episode/task indices. Episode boundaries include approach, grasp, transport, release, and the configured terminal behavior. The roadmap metadata also records whether the post-placement sequence leaves the piece in place or undoes the move for continued collection.

Collection design

Task coverage

The release covers the full Cartesian product of two piece identities and nine board cells. Counts are close to balanced but not perfectly uniform.

Task count varies from 8 to 13 episodes. Report task-macro metrics in addition to aggregate metrics so that frequent targets do not dominate evaluation.

Dataset schema

Frame-level features

The state and action axes are ordered identically:

0  shoulder_pan.pos
1  shoulder_lift.pos
2  elbow_flex.pos
3  wrist_flex.pos
4  wrist_roll.pos
5  gripper.pos

The values are the serialized position units emitted by the SO-101/LeRobot recording stack. They must not be interpreted as SI radians or as calibrated Cartesian quantities without checking the consuming robot configuration and calibration files.

Video encoding

Both image streams are stored as 640 × 480 RGB video at 30 FPS with AV1 encoding, yuv420p pixel format, and no audio. The metadata declares a GOP length of 2, CRF 30, and PyAV as the video backend.

<p align="center"> <img src="./assets/camera-pairs-episode-012.jpg" alt="Synchronized top and wrist camera samples from episode 12" width="96%"> </p>

The panel above samples six synchronized timestamps from episode 12: red X to bottom-left, 797 frames, 26.57 s. The top row is the fixed top camera; the bottom row is the wrist camera. It illustrates why the observations are complementary: one resolves global board state while the other resolves grasp and release geometry.

<p align="center"> <img src="./assets/synchronized-demonstrations.gif" alt="Synchronized top and wrist camera demonstrations" width="96%"> </p>

Temporal profile

The 195 episode durations have the following empirical distribution:

Variable duration reflects differences in source-piece pose, target-cell geometry, board occupancy, grasp correction, and post-placement behavior. Sequence-aware training and evaluation should preserve episode boundaries; frame-random splitting would leak adjacent visual and control states across train and test sets.

Joint-space diagnostics

<p align="center"> <img src="./assets/joint-trajectory-episode-012.png" alt="Six-axis state and action trajectories for representative episode 12" width="96%"> </p>

The representative trace compares the recorded state and target action for all six channels. It is a descriptive trajectory diagnostic, not a controller-identification experiment.

Across all 144,723 frames, the simultaneous difference action[t] - observation.state[t] is:

<p align="center"> <img src="./assets/action-state-gap.png" alt="Dataset-wide simultaneous action and state gap by axis" width="90%"> </p>

Interpretation boundary: these values use serialized position units and compare fields at the same recorded timestamp. They conflate target motion, controller latency, sampling alignment, mechanical response, and measurement noise. They are not lag-compensated servo tracking errors and should not be used as actuator specifications.

Image statistics

meta/stats.json contains normalized-channel statistics computed over 27,472 sampled images per camera. Values below are in [0, 1] and ordered RGB.

These statistics describe this capture environment only. They do not establish robustness to lighting, exposure, white balance, camera replacement, or background changes.

Storage layout and footprint

The LeRobot v3 paths are parameterized as:

data/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet
videos/{video_key}/chunk-{chunk_index:03d}/file-{file_index:03d}.mp4
meta/episodes/chunk-{chunk_index:03d}/file-{file_index:03d}.parquet
meta/info.json
meta/stats.json
meta/tasks.parquet
meta/hashtag_episode_plan.jsonl

Loading the dataset

Pin the revision in reproducible experiments:

from lerobot.datasets.lerobot_dataset import LeRobotDataset

DATASET_ID = "HashtagRobotics/tic-tac-toe-so101-block-a-clean-v1"
REVISION = "b1a5e8681619bd5352c29f0261843828503f1643"

dataset = LeRobotDataset(
    DATASET_ID,
    revision=REVISION,
    download_videos=True,
)

sample = dataset[0]
print(sample["observation.state"].shape)       # torch.Size([6])
print(sample["action"].shape)                  # torch.Size([6])
print(sample["observation.images.top"].shape)
print(sample["observation.images.wrist"].shape)

For metadata-only inspection, download meta/ and the Parquet shards first; video accounts for more than 99% of the repository footprint.

Recommended benchmark protocol

This release has only a published training split. A credible downstream benchmark should create and preserve a separate evaluation artifact.

1. 1.Split at the episode or game-group level. Never split individual frames randomly.
2. 2.Keep physical trials outside the training release. Re-record evaluation episodes after freezing the policy.
3. 3.Stratify by piece and target cell. Report macro averages over all 18 tasks.
4. 4.Control the initial state. Record source pose, board pose, board occupancy, lighting, camera transforms, calibration revision, and print revision.
5. 5.Use multiple independent trials. A single completed game is qualitative evidence, not a success-rate estimate.
6. 6.Report failure taxonomy. Separate perception/selection, approach, grasp, transport, release, cell accuracy, collision, timeout, and human intervention.

Suggested physical metrics:

Report trial counts and confidence intervals with every rate. Do not infer physical success from training loss or the action/state diagnostics above.

Integrity and provenance notes

• —meta/info.json, the current Parquet shards, and the pinned Hub revision define the authoritative 195-episode release.
• —meta/hashtag_episode_plan.jsonl records the 15-game roadmap and per-episode task lineage.
• —meta/hub_upload_provenance.json predates the final expansion and still describes an earlier 65-episode / 43,839-frame upload. It must not be used as the current dataset-size source.
• —The public repository currently declares one split, train: 0:195; no official validation or test artifact is included.
• —The card metrics were computed from repository metadata and frame-level Parquet values at the pinned revision. Camera panels use decoded media from the same revision.

Intended use

Appropriate uses include:

• —supervised fine-tuning of vision-language-action or imitation-learning policies;
• —study of dual-view tabletop manipulation;
• —task-conditioned action prediction for the matching six-axis SO-101 schema;
• —offline sequence modeling, representation analysis, and dataset diagnostics; and
• —reproducible development against the associated open-source system.

The dataset is not sufficient evidence for autonomous deployment and is not intended as a generic manipulation benchmark, a safety-certified control corpus, a human-interaction dataset, or a substitute for robot-specific calibration and guarded physical evaluation.

Limitations and known distribution shifts

• —One robot family, one physical scene, one board design, and one piece design.
• —Two fixed camera roles and a task-specific camera geometry.
• —Limited lighting, background, print, gripper, and calibration diversity.
• —Language coverage is templated to 18 placement commands.
• —No negative examples, adversarial states, explicit collision labels, or recovery labels.
• —No official held-out evaluation split and no repeated-trial physical benchmark.
• —Temporal correlation is high within each episode.
• —Action and state units are implementation-level serialized positions, not an SI kinematic description.

License status

No dataset license is declared in the repository metadata at this revision. Public availability does not by itself grant reuse rights. Before redistributing or using the data beyond inspection, obtain and follow an explicit license from Hashtag Robotics. The associated model is separately published under Apache-2.0; that model license does not automatically license this dataset.

Citation

If you use this dataset, cite the exact revision:

@dataset{hashtagrobotics_so101_tictactoe_2026,
  author       = {{Hashtag Robotics}},
  title        = {SO-101 Tic-Tac-Toe Block A: Games 1--15},
  year         = {2026},
  publisher    = {Hugging Face},
  url          = {https://huggingface.co/datasets/HashtagRobotics/tic-tac-toe-so101-block-a-clean-v1},
  note         = {Revision b1a5e8681619bd5352c29f0261843828503f1643}
}

References

• —End-to-end project repository
• —Readable project story
• —Associated SmolVLA policy
• —LeRobot
• —LeRobot SO-101 documentation
• —SmolVLA paper
• —Strands Agents
• —Strands Robots
• —Hashtag Robotics