sungkyunner/univtac-bspline-budgeted-routing

UniVTAC shared-budget B-spline knot routing

This repository is a no-training diagnostic for multimodal B-spline representations on four UniVTAC episodes. It fits action and tactile marker-displacement splines separately while forcing them to share one temporal knot budget.

Experiment definition

• —Tasks: Lift Bottle, Lift Can, Insert HDMI, Pull Out Key; episode 0 from each task.
• —Inputs: 8-D joint action and 4,800-D marker displacement only. No tactile RGB is used.
• —One shared budget: 16 unique temporal knot locations over the complete episode.
• —Two boundary knots are fixed; 14 interior knots are allocated greedily.
• —Cubic clamped splines produce 18 control-point vectors per modality.
• —Tolerances: action epsilon 0.002; tactile epsilon 0.1100096512 px.
• —Importance at a saved sample is max(action_error/action_epsilon, tactile_error/tactile_epsilon)^2.
• —The interval with the greatest integrated importance wins the next knot; the knot is placed at its residual-weighted median.
• —A knot is labelled action or tactile according to which modality has the larger epsilon-normalized residual at insertion. Differences within 10% are labelled mixed.

The tactile epsilon is scale-calibrated from these four episodes: marker coordinates span 55.0048 px, so the paper's approximately 0.2% relative action tolerance gives 0.002 × 55.0048 = 0.11001 px. As a sanity check, the pooled 99th percentile of coordinate changes inside recorded delay phases is 0.09265 px, just below that tolerance.

The paper's 16-row model output describes a local future spline segment and includes support-knot rows. This diagnostic intentionally interprets 16 as unique time locations over one complete episode. It tests the shared-budget idea transparently; it is not an exact reproduction of the policy's training target tensor.

Results

[image]

The temporal representation is not large relative to the approximately 60 Hz saved trajectories: each control-point vector summarizes 6.5–17.2 saved samples. The dimensional cost is asymmetric, however: 18 action control points contain 144 scalars, whereas 18 tactile control points contain 86,400 scalars. A learned or fixed tactile latent is therefore the natural next experiment.

Tactile residuals win most allocations under these epsilons. This means tactile is harder to approximate relative to its tolerance; it does not establish that tactile causally generated the event. Because the score is the maximum over 4,800 tactile coordinates but only eight action coordinates, isolated tactile outliers can also win disproportionately; marker-wise RMS or quantile routing is an important follow-up ablation.

Repository contents

• —results/*__knot_allocation.png: allocation source, greedy rank, action change, tactile magnitude, and natural-language phase.
• —results/*__action_spline.png: all eight action channels with action control points.
• —results/*__tactile_spline.png: four high-variance marker coordinates and tactile control points, separate from action.
• —results/knot_assignments.csv: every knot's time, source, selection rank, normalized residuals, and phase.
• —results/*__budget16_splines.npz: exact clamped knot vector plus action/tactile control-point arrays.
• —results/summary.json and results/REPORT.md: machine-readable and human-readable summaries.
• —src/univtac_bspline/budgeted.py: shared-budget allocator.
• —scripts/run_budgeted_routing.py: end-to-end reproduction script.

In both spline figures, circles are control points and squares are shared temporal knots placed directly on the fitted curve. Square colour indicates the allocation source: orange is action, blue is tactile, purple is mixed, and black is an episode boundary. The second x-axis gives the natural-language episode situation.

Raw UniVTAC HDF5 files are intentionally not duplicated. Download them from the original UniVTAC dataset, then run:

python scripts/download_pilot_data.py
python scripts/run_budgeted_routing.py

Related paper: B-Spline Policy: Accelerating Manipulation Policies via B-spline Action Representations.

No-training tactile phase-control follow-up

results/offline_phase_control/ replays Insert HDMI episode 0 with the fitted action B-spline while recorded marker features control the spline phase. This is an offline quasi-static test: tactile is looked up by demonstration phase, so it cannot measure counterfactual task success or prove that a jam would clear.

[image]

The mechanism successfully generated a hold and a short reversal, but the simple heuristic was not beneficial: duration increased from 1.933 s to 2.383 s, time replaying above-threshold tactile phases increased from 0.508 s to 1.267 s, and jerk RMS increased from 20.19 to 24.55. This is evidence that a smooth action spline alone does not make switched phase control smooth; phase-rate derivatives must also be constrained, and physical benefit requires closed-loop simulation.