bingaochen/Astra-on-RoboMME-Monitor

Astra-on-RoboMME Monitor

A LoRA adapter for Qwen3-VL-4B-Instruct that monitors whether the current robotic subgoal is complete from causal visual observations. It serves as the visual completion monitor in Astra-on-RoboMME. It outputs exactly true or false; it does not generate robot actions.

Files and base model

• —adapter_model.safetensors: the original trained adapter tensors (132,195,448 bytes).
• —adapter_config.json: LoRA configuration; the machine-specific base-model path was replaced with Qwen/Qwen3-VL-4B-Instruct. Tensor bytes are unchanged.
• —additional_config.json: preserved MS-Swift adapter metadata.
• —SHA256SUMS: checksums of those three inference files.

The base model is not included. Download it from Qwen/Qwen3-VL-4B-Instruct. The inference package pins base revision ebb281ec70b05090aa6165b016eac8ec08e71b17 at packaging time. The historical GCP base snapshot has not been independently compared byte for byte with that revision.

Exact input contract

Each request contains the task name, natural-language task instruction, current grounded subgoal, and 10 images in this order:

1. 1.Eight front-camera observations, oldest to newest, sampled every three environment steps: max(0, t-21), max(0, t-18), ..., t.
2. 2.The front-camera observation when the current command instance was issued.
3. 3.The current wrist-camera observation.

Early windows repeat execution frame zero where necessary. Reissuing even the same command text starts a new command instance and reference image. There is no explicit command duration, overall timestep, completed-subgoal list or relative frame-offset text in the monitor input. The text templates and image ordering must match examples/champ/input_contract.py in the code repository.

Download and load

Install the simulator/monitor environment using the code setup instructions. The original inference stack uses Python 3.11, PyTorch 2.9.1, Transformers 4.57.3, MS-Swift 3.11.1, PEFT 0.18.1 and FlashAttention 2.8.3. Keep it separate from the JAX/VLA environment. A compatible CUDA GPU and FlashAttention build are required.

from huggingface_hub import snapshot_download

adapter = snapshot_download(
    repo_id="bingaochen/Astra-on-RoboMME-Monitor",
    local_dir="checkpoints/Astra-on-RoboMME-Monitor",
)
base = snapshot_download(
    repo_id="Qwen/Qwen3-VL-4B-Instruct",
    revision="ebb281ec70b05090aa6165b016eac8ec08e71b17",
    local_dir="checkpoints/Qwen3-VL-4B-Instruct",
)

For a frozen evaluation, use the monitor revision pinned in the code repository's examples/champ/weights.json rather than an unpinned latest download. That manifest also verifies all adapter file hashes.

From the cloned code repository root, load the original inference implementation:

import os
import sys
os.environ["USE_HF"] = "1"
os.environ["IMAGE_MAX_TOKEN_NUM"] = "128"
sys.path.insert(0, "examples/champ")
from runner import Monitor

monitor = Monitor(base=base, adapter=adapter)

This applies the original patch-embedding fix and loads the adapter through MS-Swift PtEngine, with bf16, FlashAttention 2, max_tokens=8, and temperature 0. Monitor.predict(...) builds the causal ten-image request, saves the input and response, and strictly parses the boolean output. Use the supplied runner for closed-loop evaluation. A generic chat prompt or default Transformers pipeline is not a validated substitute for this input/decoding contract.

Training

Training uses recorded RoboMME demonstration-stage supervision. Ambiguous stage boundaries and conflicting identical visual inputs are excluded. Terminal-stage labels use recorded task-list completion; StopCube waiting labels use approach events. The held-out demonstration split is distinct from the benchmark val split used for closed-loop evaluation. The code release contains training settings and the input contract; dataset regeneration from raw HDF5 demonstrations is not yet fully packaged.

Pipeline evaluation

Follow the code repository README to reproduce the current pipeline on the official test split using your own GPT API credentials. All task prompts and evaluation commands are included.

License and attribution

Apache-2.0, matching this model repository's license declaration. The Qwen base, RoboMME environment and pretrained VLA retain their respective upstream licenses and attribution. See the code repository for those dependencies and citations.