Gyubeum/Qwen3-VL-8B-Instruct-UI-Genie-scoring-128k-balanced-bt-naive

Qwen3-VL-8B-Instruct-UI-Genie-scoring-128k-balanced-bt-naive

A scalar reward model for mobile GUI agent steps. Bradley-Terry fine-tune of a UI-Genie-trained Bradley-Terry stage-1 base, using Latent Peer Voting (LPV) labels built from the *_all_naive construction.

What it is

• —Base: qwen3vl_8b_128k_balanced_bt, a 128k-balanced UI-Genie BT reward model over Qwen/Qwen3-VL-8B-Instruct.
• —Objective: Bradley-Terry on LPV labels, all_naive label construction, generator weights average.
• —Checkpoint: epoch 2 of 2, LoRA rank 64 over all linear layers with the scalar score head in modules_to_save; merged here.
• —Input contract: ui-genie-rm-paired-v1 - a mobile_use tool-spec system turn carrying the logical screen size, the executed history with the recent screenshots interleaved, and the candidate action as a closed <tool_call>. Training and evaluation used this same contract, 16,384 tokens, no truncation.

What naive means here

LPV scores a candidate with its peers' likelihoods. The naive construction does not: the reward is the weight of the model that generated the candidate, so no peer log-probability is read at all. It is the generator-prior end of the study's axis, and on this corpus it is the arm that beat the full peer vote.

Usage

Score is score_head(last_token_hidden_state); higher is better. Coordinates must be in the logical-screen pixel space declared by the system prompt.

Evaluation

AndroidFlux recovery, the 122-pair android_flux_recovery_action_preference v3 export (35 clean_path, 87 error_path, 2-5 history screenshots per side), and the UI-Genie 1,000-pair set. Both are scored under the same system prompt the base was trained on.

Read these with the caveats. No recovery difference here is statistically significant - 122 pairs carry a standard error of about 4.5 - while the UI-Genie change is (n=1000, standard error about 9 pairs) and it is a loss. Every LPV fine-tune tested on this corpus sits on one tradeoff line: recovery pairs are bought with UI-Genie pairs, at roughly 6 UI-Genie pairs per recovery pair.

The score head moves very little during fine-tuning; the behaviour change comes from the merged LoRA.