data-for-agents/insta-150k-v1

InSTA: Towards Internet-Scale Training For Agents

Brandon Trabucco (1) Gunnar Sigurdsson (2) Robinson Piramuthu (2) Ruslan Salakhutdinov (1)

(1) Carnegie Mellon University, Machine Learning Department (2) Amazon

This dataset, presented in the paper Towards Internet-Scale Training For Agents, contains 150k web navigation tasks generated to facilitate Internet-scale training of agents without relying heavily on human annotations. The dataset is split into training and testing sets.

website | paper | code

Dataset Structure

The dataset contains two splits: train and test. Each example consists of two fields:

• —domain: The domain of the website (e.g., wikipedia.org).
• —task: The navigation task to be performed on the website (e.g., "Find information about the history of coffee").

The dataset can be loaded using the following code snippet:

import datasets
datasets.load_dataset("data-for-agents/insta-150k")

Evaluating Your Agents

We provide our official Llama-3.1-70B LLM Judge in our code repository, which can be used to evaluate your own agents and compare to ours. Load the train split for training your agents, and load the test split for testing purposes.

Leaderboard

To submit your own models to our leaderboard, please contact brandon@btrabucco.com. You will be asked to submit your traces on our test set. Otherwise, please follow the evaluation steps described in our paper, and use our official LLM judge to report performance numbers in your own paper.

Citing Us

Please cite our work using the following bibtex:

@misc{Trabucco2025InSTA,
  title={InSTA: Towards Internet-Scale Training For Agents},
  author={Brandon Trabucco and Gunnar Sigurdsson and Robinson Piramuthu and Ruslan Salakhutdinov},
  year={2025},
  eprint={2502.06776},
  archivePrefix={arXiv},
  primaryClass={cs.LG},
}