Datasetpublic

AndeyTait/JevForge-Mind2Web

JevForge Mind2Web Gold Decisions Private research snapshot of gold candidate-decision records used by JevForge. What is included Each JSONL record contains a page state, a choice question over candidate elements, a noul question about one candidate, complete gold target distributions, a website group, and source annotation metadata. Split Records Websites train 4,642 49 dev 786 6 calibration 400 6 test 800 8 ood 386 4 The 7,014 record IDs… See the full description on the dataset page: https://huggingface.co/datasets/AndeyTait/JevForge-Mind2Web.

sourceHugging Facecc-by-4.0updated 3d agoView on Hugging Face

0likes46downloads

Dataset Card

JevForge Mind2Web Gold Decisions

Private research snapshot of gold candidate-decision records used by JevForge.

What is included

Each JSONL record contains a page state, a choice question over candidate elements, a noul question about one candidate, complete gold target distributions, a website group, and source annotation metadata.

Split	Records	Websites
train	4,642	49
dev	786	6
calibration	400	6
test	800	8
ood	386	4

The 7,014 record IDs and website groups are disjoint across splits. The records/all.jsonl file contains the same records combined for conversion or augmentation workflows.

Important split semantics

All records are derived from the public train split of LangAGI-Lab/Mind2Web-axtree-cleaned-lite. JevForge deterministically assigns websites to its own internal splits.

Therefore:

—test is a JevForge held-out-site split, not an official Mind2Web test set;
—ood is a second held-out-site challenge split, not the official Mind2Web test-domain benchmark;
—no restricted, separately downloaded Mind2Web test archive is included.

Gold-only scope

This repository contains the direct gold conversion only:

—positive Mind2Web candidates share the action target probability mass;
—sampled negative candidates receive zero action target mass;
—the noul target is derived deterministically from candidate membership;
—no generated score labels, raw generation replies, or model-service credentials are included.

Reproduce the conversion

Source dataset revision:

text

LangAGI-Lab/Mind2Web-axtree-cleaned-lite
080a4b67d0746270166a57f2a4096a6b4129523c

Converter defaults:

text

seed=17
max_candidates=12
state_chars=2600
desc_chars=88
caps=6000,800,400,800,800
split unit=website

See conversion_manifest.json, SOURCE.json, and checksums.sha256 for the snapshot metadata.

License and attribution

Mind2Web is licensed under CC BY 4.0. This transformed snapshot retains that license and must be used with attribution to the original Mind2Web authors. The JevForge conversion code is separately licensed under MIT.

Please cite the original Mind2Web work:

bibtex

@misc{deng2023mind2web,
  title={Mind2Web: Towards a Generalist Agent for the Web},
  author={Xiang Deng and Yu Gu and Boyuan Zheng and Shijie Chen and Samuel Stevens and Huan Sun and Yu Su},
  year={2023},
  eprint={2306.06070},
  archivePrefix={arXiv},
  primaryClass={cs.CL}
}

The upstream project asks users not to redistribute its separately downloaded official test archive. This dataset does not contain that archive.