Modelpublic

drunksu/GUISwiper

sourceHugging Faceapache-2.0updated 29d agoView on Hugging Face

0likes34downloads

Model Card

GUISwiper 3B (RL)

GUISwiper is an RL-aligned GUI agent model for human-like swipe execution, introduced in the paper SwipeGen: Bridging the Execution Gap in GUI Agents via Human-like Swipe Synthesis (ACM MM 2026 Oral).

This repository hosts the final RL-aligned 3B checkpoint (bfloat16), fine-tuned from `Qwen/Qwen2.5-VL-3B-Instruct` and evaluated on SwipeBench.

Model Details

Property	Value
Base model	Qwen/Qwen2.5-VL-3B-Instruct
Parameters	3B (bfloat16, ~7 GB)
Training	RL alignment
Evaluation	SwipeBench
Input	GUI screenshots / screen videos + instruction
Output	Human-like swipe action (trajectory)
Hardware	NVIDIA GPUs (see paper for details)

Usage

python

import torch
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor

repo_id = "drunksu/GUISwiper"

model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    repo_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
processor = AutoProcessor.from_pretrained(repo_id)

# image (GUI screenshot) + instruction -> swipe trajectory
# (follow the prompt format in the SwipeGen repo for the full inference pipeline)
image = load_your_gui_screenshot()  # PIL.Image
messages = [{"role": "user", "content": [
    {"type": "image", "image": image},
    {"type": "text", "text": "Describe the swipe to perform here."},
]}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=256)
print(processor.batch_decode(output, skip_special_tokens=True)[0])

Requires transformers >= 4.49.0. See the SwipeGen GitHub repo for the complete inference and evaluation pipeline.

Download a Single File

python

from huggingface_hub import hf_hub_download

path = hf_hub_download("drunksu/GUISwiper", "model-00001-of-00002.safetensors")

Citation

bibtex

@misc{swipegen2026,
  title  = {SwipeGen: Bridging the Execution Gap in GUI Agents via Human-like Swipe Synthesis},
  author = {SwipeGen Team},
  journal = {arXiv preprint arXiv:2601.18305},
  year   = {2026},
  note   = {Code and models: \url{https://github.com/TSKGHS17/SwipeGen}}
}

If you use GUISwiper, please also reference the official repository: <https://github.com/TSKGHS17/SwipeGen>.

License & Disclaimer