Kaz55/cable-representation-analysis

Cable representation analysis — ACT encoder, DG-5F / UR5e cable sorting

What the ACT encoder's internal representation holds about the cable being handled. All figures come from a forward hook on policy.model.encoder: the transformer tokens are mean-pooled into a single 512-d vector per frame.

Every split in this directory is episode-level. Frames inside one episode are near-duplicates, so a frame-level split leaks the answer and inflates every number below.

Model under analysis

Cable naming: 1.70 sq → 1.7 mm, blue → 2.6 mm, 3.30 sq → 4.5 mm (mm = outer diameter; sq = conductor cross-section).

Headline results

The encoder knows which cable it is holding.

full 512-d linear probe, 3 cables       97.9%   (chance 33.3%)
full 512-d linear probe, 4 cable sizes  94.0%   (chance 25.0%)

It encodes thickness as a quantity, not just as a class label. Ridge regression onto physical diameter, fitted on training episodes only — the ordering is never supplied, yet 2.6 mm lands between 1.7 and 4.5:

held-out R^2 = 0.88
  true 1.7 mm -> 1.81 +- 0.33 mm
  true 2.6 mm -> 2.66 +- 0.39 mm
  true 4.5 mm -> 4.33 +- 0.43 mm

But the discrimination is visual, not tactile. Splitting each episode at contact onset (detected per episode from finger torque) gives almost the same accuracy before the GelSight has touched anything as after:

             probe accuracy   diameter R^2
pre-contact      94.9%            0.72
post-contact     95.6%            0.83

This reconciles the training-loss result seen across 4+ datasets: removing GelSight entirely leaves training loss unchanged, and GelSight resolution from 192 tokens down to 0 makes no difference. The policy separates cables by sight.

PCA in each phase, probe accuracy on exactly the axes drawn:

             PC1/PC2   PC1-PC3      variance PC1 / PC2 / PC3
pre-contact   46.7%     77.2%          18.9% / 9.2% / 5.8%
post-contact  64.9%     82.8%           9.7% / 7.0% / 6.1%

PC1/PC2 is dominated by arm pose, which is why the unsupervised 2-D view looks messier than the probe numbers suggest. The cable-identity structure lives on later components (PC3/PC4) and in the 3-D view.

Four cable sizes — does an intermediate size land in between?

4sizes/. Adds 3.00 sq, an intermediate cross-section, from Kaz55/dg5f_ur5e_cable3sizes_180ep. 900 frames per size (60 episodes × 15 frames), 3600 total.

Yes — 3.00 sq sits between blue and 3.30 sq. On the regressed size axis:

size axis held-out R^2 = 0.86
  1.70 sq        true 1.70 -> 1.83 +- 0.22
  blue (2.6 mm)  true 2.30 -> 2.36 +- 0.22
  3.00 sq        true 3.00 -> 2.94 +- 0.22
  3.30 sq        true 3.30 -> 3.21 +- 0.20

and in raw 512-d centroid distances, where blue↔3.30 is the largest of the three — exactly what it must be if 3.00 lies between them:

  blue <-> 3.00   4.59
  3.00 <-> 3.30   3.33
  blue <-> 3.30   5.27   <- largest

Caveat. 3.00 sq comes from a different dataset, so session-level confounds (lighting, gel condition, recording day) travel with that label. The ordering result is what guards against over-reading it: a session artefact has no reason to land between two other sizes — it would sit off to one side.

blue was entered as 2.30 sq for the ordering test; the ordering itself does not depend on that value.

Directory map

Reproducing

Replot from the cached features (no GPU, no model download):

python scripts/plot_4sizes_tsne.py --cache cache/cache_4sizes.npz --outdir 4sizes/
python scripts/plot_contact_pca.py --cache cache/cache_cable_contact.npz --out contact/cable_contact_pca.png
python scripts/summarize_from_cache.py --cache cache/cache_cable_feats.npz

Re-extract features from the trained policy (needs the checkpoint and a GPU):

python scripts/collect_4sizes.py --policy-path <checkpoint>
python scripts/collect_contact_feats.py --policy-path <checkpoint>