Lexmount/LexBench-Browser

1---2language:3  - zh4  - en5license: cc-by-4.06task_categories:7  - question-answering8  - text-generation9  - reinforcement-learning10pretty_name: LexBench-Browser11size_categories:12  - n<1K13tags:14  - browser-agent15  - web-agent16  - benchmark17  - evaluation18  - automation19---20 21# LexBench-Browser22 23LexBench-Browser is a public browser-agent dataset for evaluating agents on real-web workflows.24The v1.0 snapshot contains 210 no-login tasks across 107 distinct websites, with Chinese and25English instructions, task-level reference steps, key points, common mistakes, scoring rubrics,26and robustness tags.27 28Repository: https://github.com/lexmount/browseruse-agent-bench29 30Dataset page: https://huggingface.co/datasets/Lexmount/LexBench-Browser31 32Docs: https://docs.bubench.lexmount.io/33 34Contact: lexbench@lexmount.com35 36## Dataset Summary37 38LexBench-Browser is designed to support browser-agent engineering in browseruse-agent-bench:39 40- run an agent against real websites41- compare local and cloud browser backends42- evaluate task success with a declared judge strategy43- inspect trajectories and failure modes44- submit reproducible leaderboard results45 46The dataset does not require login for the v1.0 public snapshot.47 48## Files49 50```text51LexBench-Browser/52|-- data_info.json53|-- task.jsonl54|-- task_global.jsonl55|-- task_lexmount.jsonl56`-- VERSION_HISTORY.md57```58 59Splits:60 61| Split | File | Tasks | Notes |62| --- | --- | ---: | --- |63| `All` | `task.jsonl` | 210 | Default public v1.0 split |64| `global` | `task_global.jsonl` | 92 | Global-region task subset |65| `lexmount` | `task_lexmount.jsonl` | 118 | Lexmount-region task subset |66 67## Fields68 69Each JSONL row includes:70 71- `id`: stable task id72- `query`: user-facing browser task73- `task_type`: task type label74- `domain`: domain category75- `difficulty`: `easy`, `medium`, or `hard`76- `login_required`: whether login is required77- `risk_control`: whether the task has risk-control constraints78- `target_website`: intended website or website family79- `reasoning_type`: reasoning complexity label80- `language`: `zh` or `en`81- `website_region`: expected region/language context82- `reference_answer`: reference steps, key points, common mistakes, and scoring rubric83- `score_threshold`: pass threshold84- `robustness_tags`: practical browser-agent stressors85 86## Label Distribution87 88Language:89 90| Language | Tasks |91| --- | ---: |92| `zh` | 137 |93| `en` | 73 |94 95Reasoning type:96 97| Reasoning type | Tasks |98| --- | ---: |99| `single_step` | 117 |100| `multi_step` | 70 |101| `deep_analysis` | 23 |102 103Domain:104 105| Domain | Tasks |106| --- | ---: |107| `finance_gaming` | 44 |108| `video_platform` | 42 |109| `tools_education` | 40 |110| `general` | 34 |111| `social_lifestyle` | 26 |112| `ecommerce` | 23 |113| `gaming` | 1 |114 115## Robustness Tags116 117Tags cover:118 119- popup interference: `login_popup`, `cookie_consent`, `ad_overlay`120- sequence complexity: `long_sequence`, `deep_navigation`, `multi_site`121- content dynamics: `realtime_data`, `lazy_load_scroll`, `iframe_embed`122- anti-crawl behavior: `captcha_verification`, `anti_bot`, `rate_limiting`123- localization: `chinese_rendering`, `cross_language`124- complex interaction: `filter_sort`, `data_extraction`125 126## Usage127 128```bash129git clone https://github.com/lexmount/browseruse-agent-bench.git130cd browseruse-agent-bench131uv sync --extra browser-use132uv run bubench run --agent browser-use --data LexBench-Browser --mode first_n --count 3133```134 135For official leaderboard submissions, follow the repository evaluation protocol and result136submission docs.137 138## Licensing139 140The LexBench-Browser dataset metadata and task definitions are released under CC-BY 4.0.141 142This license covers the curated task records, labels, reference steps, scoring rubrics, and143metadata authored for the dataset. It does not relicense third-party website content,144screenshots, traces, marks, page text, or other artifacts collected while running agents.145 146Repository code is licensed separately under Apache-2.0.147 148## Limitations149 150- Real websites change over time, so tasks may become easier, harder, or temporarily unavailable.151- Some websites may show region-specific content, anti-bot interstitials, cookie banners, or152  localized layouts.153- The public v1.0 snapshot avoids login-required tasks.154- Automated judge results should be interpreted with the declared judge model, prompt strategy,155  benchmark version, and browser backend.156 157## Citation158 159Use the repository `CITATION.cff` for citation metadata.160