FineEnvs/geoguesser-tasks

GeoGuesser Task Splits

Task indexes for the GeoGuesser OpenEnv environment. Each line is one episode: an ordered list of panorama frames with coordinates, headings and capture dates, plus the sequence and contributor it came from.

What a task is

These files carry metadata only, not imagery. Every frame's coordinates, heading and capture date are here, so the movement graph resolves with no network access; only image bytes need fetching, and Mapillary's thumb_*_url values are expiring signed CDN URLs that cannot be stored. Resolve them from image_id through the Mapillary Graph API, or mirror them once with scripts/build_tasks.py from the environment repository.

How the split was made

Both splits are carved from one 3,673-task pool, so contamination is enforced exactly once rather than reasoned about across two separate harvests. The rules follow the OSV-5M paper, which built its train/test split from the same Mapillary source:

• —no shared sequence_id between splits
• —no training task within 1 km of an eval task

The buffer matters because frames sit about 3.3 m apart: holding out an image while keeping its neighbour holds out nothing. The split script verifies its own output and exits non-zero if either rule is violated.

Eval is carved first, balanced by country and capped at 4 tasks each, because at a couple of hundred tasks the balance decides what the score means. An earlier unbalanced attempt put 28% of the set in one country.

Provenance

Imagery is from Mapillary, CC BY-SA 4.0. Each task records its contributor in attribution, which the environment displays. Only 360-degree panoramas are included (camera_type == "spherical"; note that the documented value equirectangular does not appear in practice).

Sequences were discovered by enumerating Mapillary's mly1_public vector tiles at zoom 6, where the sequence layer carries is_pano — 1.2 million panorama sequences worldwide. Candidates are sampled with weight proportional to local image density raised to -0.75, the OSV-5M weighting, then capped per country and per contributor: one contributor alone holds 8% of the pool.

Reproducing

git clone https://github.com/huggingface/OpenEnv
cd OpenEnv/envs/geoguesser_env
export MAPILLARY_API_KEY_TRAIN="MLY|..."

python scripts/harvest_tiles.py                 # enumerate sequences worldwide
./scripts/build_dataset.sh                      # assemble and mirror tasks
python scripts/verify_offline.py tasks/pool_offline_5k.jsonl
python scripts/split_tasks.py tasks/pool_offline_5k.jsonl --eval 200

A rebuild will not reproduce these exact tasks — the pool is sampled and upstream coverage changes — which is precisely why the split is published rather than left to be regenerated.