victoria-kp/goodbuy-bestbuy-mirrored-tasks

Goodbuy / Bestbuy Mirrored Web Agent Tasks

320 tasks per site for training and evaluating web agents on a simulated store and its real counterpart: Goodbuy (the simulated Best Buy clone from Westworld) and the real bestbuy.com. Built for the Halluminate blog post on sim-to-real transfer for web agents.

Each site has the 20 original Westworld Goodbuy tasks plus 300 procedurally generated ones (100 per task type: checkout, delivery instructions, store pickup). Every Bestbuy task is a 1:1 mirror of a Goodbuy task, built from the same values; the mirrors field cross-links each pair, so transfer can be measured on matched task pairs.

Files

• —goodbuy_tasks.jsonl - 300 generated Goodbuy (sim) tasks, in the Westworld task schema (task_generation_config_json holds the instruction, URL template, and ground-truth values).
• —bestbuy_tasks.jsonl - 300 generated Bestbuy (real site) tasks. Each row has the natural-language task instruction, the user_info ground truth, a stopping_criterion, and the mirrors field pointing at its Goodbuy twin.
• —bestbuy_tasks_original20.jsonl - the 20 hand-written Bestbuy tasks that mirror the original Westworld Goodbuy tasks, in the same schema. Together with the generated ones this completes the 320 tasks per site that the split files reference.
• —splits/ - the train/test split files used in the blog post (80/20 per site, stratified by task type, fixed seed; the mixed_* files are the size-matched 128+128 split for the mixed training condition).

The 20 original Goodbuy (sim) tasks are in the Halluminate/westworld dataset; their 20 Bestbuy mirrors are included here as bestbuy_tasks_original20.jsonl.

Safety notes

• —All identities are fictitious. Names, addresses, emails, phone numbers, and credit card numbers are procedurally generated. Card numbers are Luhn-valid so payment forms accept them, but they are not real cards.
• —Bestbuy tasks never place an order. Every instruction stops at the point of purchase ("do NOT place the order"), and the matching verifier checks that no order was placed.

Generation

Tasks were generated by randomizing every form field the agent must type (name, shipping address, email, phone, card number, delivery dropoff location, property type, delivery notes, pickup method), constrained to values the sites actually accept, with five phrasing templates per task type. The starting cart (two items) is fixed on both sites.

Trained adapters

Adapters trained on these tasks (Qwen2.5-VL-7B, QLoRA):

Full set: collection