hugging-apps/fara-computer-use-agent

Fara1.5-9B Computer Use Agent — Visual Grounding Demo

This Space demonstrates Fara1.5-9B, a 9B multimodal computer use agent by Microsoft Research AI Frontiers.

How it works

1. 1.Upload a screenshot of a web page
2. 2.Describe the task the agent should perform (e.g. "Click the search box and type 'wireless headphones'")
3. 3.Fara1.5-9B analyzes the screenshot and predicts the next grounded action
4. 4.The predicted action (click coordinate, text to type, scroll, etc.) is visualized on the image with a marker

Technical details

• —Model: microsoft/Fara1.5-9B (9B parameters, bf16, ~18GB VRAM)
• —Base: Qwen3.5-9B, fine-tuned with FaraGen1.5 synthetic data
• —Coordinate system: Normalized 0–1000 per axis, mapped onto a 1440×900 viewport
• —Output format: Chain-of-thought reasoning followed by an XML-tagged computer_use tool call
• —System prompt: The verbatim system prompt from the model card

The model predicts pixel-level click coordinates in a normalized 0–1000 space, which are rescaled to the actual screenshot dimensions for visualization.