Kavin60606/so101_isr_193

SO-101 Teleop — ISR-Standardized (193 episodes)

Training arm B of the ISR vs Partner-sampling SmolVLA experiment. This is the client's SO-101 teleop set with ISR (Information-Standardized Trajectory Resampling) applied — pauses removed, motion kept at uniform information spacing — for comparison against the raw baseline and the partner company's length-normalization method.

TL;DR

What ISR does

ISR resamples a trajectory by keeping one frame per fixed amount of information — instead of one frame per fixed amount of time:

information(step) = λ_dist · (joint distance moved) + λ_acc · (accumulated acceleration)

• —Pauses / slow drift → ~0 information → dropped (removes the "same observation, contradictory stay-vs-move label" that makes policies hesitant).
• —Fast motion + deceleration-before-contact → information-dense → sampled densely.
• —Gripper open/close → force-kept (the grasp happens while the arm is nearly still, so it would otherwise be dropped).

No interpolation — the kept frames are the real recorded frames (pixels, state, action untouched); they are only re-indexed onto a uniform 30 fps grid.

Knobs (calibrated for SO-101, degrees)

Calibration measured on the 193 train episodes: median per-frame joint step 1.43°, mean acceleration 271, gripper span 40.9°/episode.

d_target is the compression dial in the signal's own units. It is anchored to the median joint step, so the same recipe transfers across robots/units; the paper's metre-values do not apply directly to degrees.

fps / 30 Hz alignment

All arms (baseline, ISR, partner) are stamped at 30 fps on a uniform grid. ISR keeps fewer frames, so an episode's dataset-time duration shrinks (its real-time non-uniform spacing is approximated as a uniform 30 fps grid). This keeps the SmolVLA action-chunk horizon (chunk_size=50) and the eval fps identical to the baseline, so the frame selection is the only variable.

Split

Holdout = last 3 episodes of each task string, [96,97,98,196,197,198] (1,622 frames), produced by lerobot eval_split=0.025. These are excluded from this dataset and reserved for evaluation — no arm ever trains on a holdout frame.

How it was built

data/teleop_std_poc pipeline:

1. 1.fetch_trajs.py — read source states/actions, slice SO-101 layout (5 joints + gripper).
2. 2.isr_resample.py — the ISR DP (derived from the official Apache-2.0 repo D-Robotics-AI-Lab/ISR).
3. 3.build_dataset.py / build_parallel.py — materialize a LeRobot v3.0 video dataset:
4. 4.sequential torchcodec range-decode of the source video (verified pixel-identical to lerobot's per-frame reader, ~40–90× faster), then h264 re-encode of the kept frames;
5. 5.parallel sharded across CPU cores + aggregate_datasets merge (build in ~15 min).

Training / experiment

• —Policy: SmolVLA (lerobot 0.6.0), byte-identical to the client recipe except the dataset.
• —20,000 steps, batch 16/GPU × 4, bf16, 4× RTX 3090.
• —Compared against: raw baseline angkul07/mm_SO101_teleop (teleop100) and the partner-sampled arm Kavin60606/so101_partner_193.
• —Eval: the 6-episode holdout — action MSE/MAE, per-joint MAE, gripper accuracy, and CI-MSE (grasp/place-interval error, where ISR's contact-preservation should show).

Status — trained + evaluated

Dataset built + verified (193 eps, 32,175 frames, video). SmolVLA trained 20k steps (Kavin60606/smolvla_so101_isr). Eval on the 6 holdout eps (1622 frames, seed 1000):

Result: ISR underperformed the raw baseline here. Global MSE ~1.8× worse and grasp-interval CI-MSE 2.5× worse — the interval ISR aimed to protect. Release CI-MSE held (≈ baseline). Read: at d_target=4.0 ISR stripped the low-motion contact-approach/settle frames around the grasp (gripper force-keep saves the close event but not the near-still approach), leaving a sparse, jumpy contact phase for the policy. A gentler d_target (keep ~80%) + wider gripper-window force-keep is the obvious next calibration. Not a verdict on ISR generally — a calibration result on this task.