Modelpublic

FineEnvs/geoguesser-qwen3.5-4b-grpo

sourceHugging Faceapache-2.0updated 3d agoView on Hugging Face

0likes53downloads

Model Card

GeoGuesser · Qwen3.5-4B · GRPO run 1

A LoRA adapter that plays GeoGuessr: dropped at a random street corner on Earth, it works out where it is and commits to a coordinate. Trained with GRPO against a live OpenEnv environment serving real Mapillary panoramas.

This is run 1, step 1000, and it is the adapter behind every number in the write-up.

What it scores

200 held-out tasks across 73 countries, pass@4, 800 episodes per arm, all scored through the same curve. The full board is in the project README.

model	mean-of-4	best-of-4	median error
claude-sonnet-5	0.6952	0.8397	324 km
this adapter (Qwen3.5-4B + LoRA)	0.6445	0.7092	662 km
gpt-5.4-mini	0.5732	0.7719	753 km
claude-haiku-4.5	0.5374	0.7014	939 km
Qwen3.5-122B-A10B	0.5338	0.6987	767 km
Qwen3.5-4B, untrained	0.4825	0.6589	1226 km

Against its own base, paired per task: +0.1620 ± 0.0137 (95% CI), better on 169 of 200 tasks. Country identified 21.2% → 33.1%. Episodes floored at zero on the environment's own curve 29.5% → 0.5%.

Scores are on the training reward curve, min(1, 0.5·e^(-d/1492.7) + 0.5·e^(-d/5000)) × (1 - min(cost, 0.2)), recomputed from raw distance. That is not GeoGuessr's own curve, which floors at zero past roughly 3,500 km.

How it behaves, which is the interesting part

It stopped exploring. Turns per episode fell from 6.7 to 1.1, output tokens from 1,062 to 66, and accuracy went up. It recognises the region from the first frame and commits to a plausible city in it, rather than reading signs across a dozen turns. That is not a random guess (a random guess on land averages roughly 8,000 km of error; this is 662 km) and it is not reward hacking: no identity leak in 882 audited prompts, no repeated tasks, and an action cost of 0.002 on the converged policy. It is what the reward rewarded, and the write-up is about why.

Training


base	`Qwen/Qwen3.5-4B`
method	GRPO (TRL), `environment_factory` multi-turn tool calling
LoRA	r=16, α=32, dropout 0.05, on `q/k/v/o_proj`
steps	1000, one task per optimizer step, `NUM_GENERATIONS=8`
rollouts	8,000 episodes over 1,000 distinct tasks
turns	12 max
image	448 px
optimiser	LR 3e-5, temperature 1.0, `beta=0`, `scale_rewards="group"`
hardware	4×A100 80 GB (one A100 also works), 10.2 hours, about $100
environment	`HuggingEnvs/geoguesser-env`, over HTTP

Note that it plateaued by step 200 (0.6393 against step 1000's 0.6445, inside the noise). If you reproduce this, stop at 250.

Using it

python

from peft import PeftModel
from transformers import AutoModelForImageTextToText, AutoProcessor

model = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.5-4B", device_map="auto")
model = PeftModel.from_pretrained(model, "HuggingEnvs/geoguesser-qwen3.5-4b-grpo")
processor = AutoProcessor.from_pretrained("Qwen/Qwen3.5-4B")

Or serve it with vLLM and point the eval harness at it:

bash

vllm serve Qwen/Qwen3.5-4B --enable-lora \
  --lora-modules run1=HuggingEnvs/geoguesser-qwen3.5-4b-grpo

Everything else

—[The write-up](https://huggingface.co/spaces/HuggingEnvs/geoguesser-article), which explains every decision and what it cost to learn it
—[The environment](https://huggingface.co/spaces/HuggingEnvs/geoguesser-env), playable in a browser
—[The task splits](https://huggingface.co/datasets/HuggingEnvs/geoguesser-tasks) and [the imagery](https://huggingface.co/buckets/HuggingEnvs/geoguesser-panos)
—[All four training runs](https://huggingface.co/spaces/HuggingEnvs/geoguesser-trackio) in one dashboard
—[Code and exact commands](https://github.com/adithya-s-k/HuggingEnvs/tree/main/03-geoguesser)

Imagery is Mapillary, CC BY-SA 4.0.

Citation

bibtex

@misc{fineenvs,
  author = {Kolavi, Adithya S},
  title  = {FineEnvs: Open Source RL Environments for LLM Agents},
  year   = {2026},
  url    = {https://github.com/adithya-s-k/FineEnvs}
}