Modelpublic

Dwipz/Anchor-Align

sourceHugging Facemitupdated 28d agoView on Hugging Face

2likes139downloads

Model Card

Anchor-Align: Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment

**🌐 Project Page**  •  **📄 Paper (arXiv)**  •  **💻 Code (GitHub)**  •  **🎬 Videos**

TL;DR — Standard behavior-cloning finetuning of a vision-language model on robot demos silently erases the pretrained VLM's semantics and decouples its language output from its actions. Anchor-Align adds two lightweight losses on top of action prediction that (i) anchor the trainable VLA to a frozen copy of the pretrained VLM (preserving vision-language reasoning) and (ii) align the pre-action hidden state with a discrete motion-direction label derived from the executed motion. The result: 22.6% on LIBERO-PRO position swap (vs 2.3% for standard BC), 90.3% LIBERO-Plus average (vs 85.1%), and near-doubled real-world success (28.3% → 54.2%) on a UFactory xArm7 setup.

📦 Companion GitHub Repository

This repo hosts the weights. The inference and evaluation code lives at [github.com/dwipddalal/Anchor-Align](https://github.com/dwipddalal/Anchor-Align).

	HuggingFace (`Dwipz/Anchor-Align`)	GitHub (`dwipddalal/Anchor-Align`)
Weights + tokenizer + configs	✅	—
LIBERO / LIBERO-PRO / LIBERO-Plus eval scripts	—	✅
Eval SLURM templates + 3-seed campaign runners	—	✅
Paper results tables + figures	✅	✅
Per-checkpoint `MODEL_CARD.md`	✅	—
Reproduction recipe (`REPRODUCE.md` + verifier)	—	✅

Note: the training code is not part of the initial release; we plan to release it in a follow-up update. Each checkpoint's full training configuration is documented in its MODEL_CARD.md.

Available Checkpoints

Folder	Suite	MSE weight	Steps
`libero-spatial/`	LIBERO Spatial	0.10	10k
`libero-object/`	LIBERO Object	0.15	2.5k
`libero-goal/`	LIBERO Goal	0.10	25k
`libero-long/`	LIBERO-10 (Long)	0.15	45k

All four finetune Prismatic Qwen2.5-0.5B (DINOv2 + SigLIP, 24 layers) with LoRA rank 64 and an L1-regression MLPResNet action head. Per-checkpoint metrics live in each subfolder's MODEL_CARD.md; reproduction targets and tolerances are in the GitHub repo's `REPRODUCE.md`.

Results

Tables below are reproduced from the paper. Reproduction targets and tolerances for the released checkpoints are in `REPRODUCE.md`.

Standard LIBERO suites

Success rates on the four standard (unperturbed) LIBERO suites. Anchor-Align achieves the highest success rate on every suite, surpassing methods with substantially larger backbones and large-scale robot-action pretraining.

Method	Spatial	Object	Goal	Long
Diffusion Policy	78.3	92.5	68.3	50.5
π₀-FAST	87.0	63.0	89.0	48.0
SmolVLA-0.24B	87.0	93.0	88.0	63.0
SmolVLA-2.25B	93.0	94.0	91.0	77.0
OpenVLA-OFT	94.3	95.2	91.7	86.5
MolmoAct	87.0	95.4	87.6	77.2
π₀.₅-KI	96.6	97.2	94.6	85.8
VLA-0	93.6	96.0	95.6	87.6
VLA-Adapter [Frozen]	89.4	89.6	88.0	84.5
VLA-Adapter (standard BC)	96.0	99.8	96.0	89.0
Anchor-Align VLA (ours)	98.4	100.0	97.2	90.8

Robustness and generalization — LIBERO-PRO and LIBERO-Plus

Success rates under perturbation on the LIBERO-Spatial suite (paper Table 1). Bold = best, underline = second best.

<table> <thead> <tr><th rowspan="2" align="left">Method</th><th colspan="4">LIBERO-PRO</th><th colspan="8">LIBERO-Plus</th></tr> <tr><th>Lang. Reph.</th><th>Object Swap</th><th>Pos. Swap</th><th>Mean</th><th>Lang. Instr.</th><th>Bg. Text.</th><th>Robot Init</th><th>Cam. View</th><th>Obj. Layout</th><th>Light Cond.</th><th>Sensor Noise</th><th>Mean</th></tr> </thead> <tbody> <tr><td align="left">Co-training + KI*</td><td>54.0</td><td>77.4</td><td>0.0</td><td>43.8</td><td>48.0</td><td>82.6</td><td>25.7</td><td>64.6</td><td>65.7</td><td>73.3</td><td>49.0</td><td>57.1</td></tr> <tr><td align="left">MolmoAct</td><td>77.8</td><td>82.4</td><td>0.0</td><td>53.4</td><td>79.5</td><td>84.1</td><td>47.4</td><td>10.1</td><td>76.5</td><td>77.4</td><td>53.4</td><td>60.8</td></tr> <tr><td align="left">OpenVLA-OFT</td><td>74.4</td><td>95.2</td><td>0.0</td><td>56.5</td><td>81.5</td><td>95.7</td><td>40.3</td><td>94.7</td><td>88.6</td><td>95.5</td><td>28.2</td><td>74.1</td></tr> <tr><td align="left">VLA-Adapter [Frozen]</td><td>56.0</td><td>73.4</td><td>0.0</td><td>43.1</td><td>41.5</td><td>70.9</td><td>35.1</td><td>94.4</td><td>62.3</td><td>84.9</td><td>36.2</td><td>59.9</td></tr> <tr><td align="left">VLA-Adapter (standard BC)</td><td>91.1</td><td>89.6</td><td>2.3</td><td>61.0</td><td>85.1</td><td>90.7</td><td>52.6</td><td>92.6</td><td>93.2</td><td>93.2</td><td>89.5</td><td>85.1</td></tr> <tr><td align="left">Anchor-Align VLA (ours)</td><td>97.0</td><td>96.2</td><td>22.6</td><td>71.9</td><td>87.2</td><td>99.6</td><td>59.1</td><td>96.3</td><td>97.4</td><td>99.0</td><td>96.9</td><td>90.3</td></tr> </tbody> </table>

\*Our implementation of knowledge insulation adapted to VLA-Adapter. Position swap is the hardest axis: MolmoAct and OpenVLA-OFT score 0%, standard BC reaches 2.3%, while Anchor-Align reaches 22.6%.

Qualitative — generalization to semantic perturbations

Same task, different phrasing / different object identity / shuffled positions. Anchor-Align retains the VLM's semantic understanding of what the instruction refers to, while the baseline latches onto memorized appearance shortcuts.

Per-suite robustness — Object, Goal, and Long

The same gains carry over to the remaining three LIBERO suites. Each radar plot compares Anchor-Align (orange) against the standard BC VLA-Adapter baseline (gray) across nine evaluation axes: two from LIBERO-PRO (Language Rephrase, Object Swap) and seven from LIBERO-Plus.

Largest gains: Robot Init State +18.6 on Object; Language Instruction +11.9 on Goal; Lighting Condition +20.8, Object Layout +18.6, and Camera Viewpoint +17.7 on Long.

Long-horizon generalization — CALVIN ABC→D

Each rollout chains five language instructions; k/5 is the fraction of rollouts completing the first k, and Len is the average number of consecutively completed tasks.

Method	1/5	2/5	3/5	4/5	5/5	Len
UniVLA	95.5	85.8	75.4	66.9	56.5	3.8
OpenVLA-OFT	96.3	89.1	82.4	75.8	66.5	4.1
OpenHelix	97.1	91.4	82.8	72.6	64.1	4.1
VLA-Adapter (standard BC)	<u>98.3</u>	<u>94.0</u>	<u>87.5</u>	<u>80.0</u>	<u>73.1</u>	<u>4.3</u>
Anchor-Align VLA (ours)	99.1	95.8	90.6	84.7	77.9	4.5

Multi-seed significance

Mean ± standard deviation over 5 training seeds on LIBERO-Spatial; the method gaps are far larger than seed-to-seed variability.

Method	Spatial (Std)	Lang. Reph.	Object Swap	Pos. Swap	Plus
VLA-Adapter (standard BC)	93.3 ± 0.3	91.1 ± 0.4	90.1 ± 0.5	2.6 ± 0.7	85.3 ± 0.3
Anchor-Align VLA (ours)	97.9 ± 0.3	97.1 ± 0.5	96.1 ± 0.4	23.5 ± 0.2	90.5 ± 0.6

Per-folder layout

Each subfolder is a self-contained inference bundle:

<benchmark>_<steps>/
  MODEL_CARD.md
  model.safetensors                        # merged base VLM weights (~2.5 GB)
  lora_adapter/adapter_model.safetensors   # separate LoRA weights (~208 MB)
  action_head--<step>_checkpoint.pt        # L1 regression head (~436 MB)
  align_dir_proj--<step>_checkpoint.pt     # alignment direction projector
  proprio_projector--<step>_checkpoint.pt  # proprioception projector
  config.json, tokenizer.*, processor_config.json, dataset_statistics.json

Quickstart — download + run inference

1. Clone the code repo (contains all inference and eval scripts):

bash

git clone https://github.com/dwipddalal/Anchor-Align.git
cd Anchor-Align
pip install -e .   # + follow the installation section in the GitHub README

2. Download a checkpoint from this repo:

python

from huggingface_hub import snapshot_download

local_dir = snapshot_download(
    repo_id="Dwipz/Anchor-Align",
    allow_patterns=["config.json", "libero-spatial/*"],   # or libero-object/*, libero-goal/*, libero-long/*
    local_dir="./checkpoints",
)

3. Run inference using the eval scripts in the GitHub repo:

bash

# LIBERO Standard
CUDA_VISIBLE_DEVICES=0 python experiments/robot/libero/run_libero_eval.py \
  --pretrained_checkpoint ./checkpoints/libero-spatial \
  --task_suite_name libero_spatial \
  --use_proprio True --num_images_in_input 2 --use_pro_version True

# LIBERO-PRO (paraphrased-language perturbation)
CUDA_VISIBLE_DEVICES=0 python experiments/robot/libero_pro/run_libero_pro_eval.py \
  --pretrained_checkpoint ./checkpoints/libero-spatial \
  --base_suite_name libero_spatial --perturbation_type lan \
  --use_proprio True --num_images_in_input 2 --use_pro_version True

Inference flags (--use_proprio True --num_images_in_input 2 --use_pro_version True --use_l1_regression True --center_crop True --num_open_loop_steps 8) match the paper. See the GitHub README's "Evaluation" section for LIBERO-Plus and batched variants.

License

MIT. See the GitHub repository LICENSE.

Citation

bibtex

@article{dalal2026anchoralign,
  title   = {Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment},
  author  = {Dalal, Dwip and Patel, Shivansh and Jain, Chahit and Kim, Jeonghwan and Mishra, Utkarsh and Baratian, Alex and Ha, Hyeonjeong and Ji, Heng and Lazebnik, Svetlana and Jain, Unnat},
  journal = {arXiv preprint arXiv:2607.13429},
  year    = {2026}
}

Keywords

Vision-language-action models, robot learning, robot manipulation, continual learning, catastrophic forgetting, out-of-distribution generalization, language-action alignment, behavior cloning, imitation learning, knowledge distillation, VLA finetuning.