sungkyunner/univtac-bspline-knot-analysis

UniVTAC × B-spline Policy fitting pilot

This repository fits cubic B-splines to an augmented target containing UniVTAC's next 8-D joint action and next 4,800-D GelSight Mini marker displacement. It deliberately performs fitting only, not policy training.

The completed pilot outputs are in `results/`. Each task/setting has:

• —a PNG of the complete episode with natural-language phase labels and action/tactile knot colors;
• —an NPZ containing the complete knot vector and every 4,808-D control point (all in raw joint/pixel units);
• —a CSV assigning each adaptive interior knot to action, tactile, or mixed residuals and to an episode situation.

See `DESIGN.md` for the proposed paper-compatible VLA architecture and important execution caveats.

Reproduce

python3 -m pip install -e .
python3 scripts/download_pilot_data.py
MPLBACKEND=Agg python3 scripts/run_pilot.py
pytest -q

The downloader retrieves episode 0.hdf5 for lift_bottle, lift_can, insert_HDMI, and pull_out_key directly from byml/UniVTAC. The four source files total about 590 MB and are ignored by git.

Target definition

UniVTAC stores each sensor's marker flow as [reference_uv, current_uv]. This pilot uses:

marker displacement = current_uv - reference_uv
augmented target[t] = concat(joint[t+1, :8], marker_displacement[t+1])

The one-step shift and first-eight-joint convention match UniVTAC's ACT conversion. Simulation steps are converted to seconds at 120 Hz; saved observations are two simulation steps apart (60 Hz in these episodes).