ByteDance-Seed/EdgeBench
Overview EdgeBench is a benchmark of 134 real-world tasks for evaluating how autonomous AI agents learn from real-world environments. Instead of measuring one-shot performance, EdgeBench places agents in executable task environments with realistic, multi-level feedback and lets them iterate for 12+ hours per task — tracking the full trajectory of improvement, not just the final score. We publicly release 51 tasks… See the full description on the dataset page: https://huggingface.co/datasets/ByteDance-Seed/EdgeBench.
847.1k
1{2 "task_id": "vibrating_path_graph_coloring",3 "name": "Vibrating Path Graph Coloring",4 "category": "Combinatorial Optimization",5 "base_image": "python310",6 "platform": "linux/amd64",7 "internet": false,8 "cwd": "/home/workspace/vibrating_path",9 "submit_paths": [10 "solution.py"11 ],12 "work": {13 "image_tag": "e5597c7f3c12",14 "specs_dir": "/home/workspace/vibrating_path",15 "agent_query": "## Vibrating Path - Graph K-Coloring (CodeChef VBR)\n\nWrite `solution.py` in the project root that reads from stdin and writes to stdout.\n\n---\n\n## Problem Overview\n\nRead `README.md` and `tools/README.md` for full problem details. A baseline `solution.py` already exists (it produces syntactically valid but low-quality output). Your job is to improve it.\n\n---\n\n## Evaluation\n\nYour solution is scored on **50 fixed test cases**. Final score = sum of individual case scores. **Lower is better.**\n\n---\n\n## Local Testing\n\nGenerate local random tests with `./tools/bin/gen <seed>`, using seeds in the range **0..10000** only.\n\n```bash\n# Generate a random test case (seed-based, deterministic)\n./tools/bin/gen 1 > input.txt\n\n# Run your solution\npython3 solution.py < input.txt > output.txt\n\n# Score output (Lower is better)\n./tools/bin/tester input.txt output.txt\n# Outputs to stderr: Score = <N>\n```\n\n---\n\n## Rules\n\n- Write your solution as `solution.py` in the project root directory\n- Do NOT modify files in `tools/`\n- Use `tools/bin/gen` and `tools/bin/tester` for local testing\n- Your program should read from stdin and write to stdout\n- Run your solution to completion and verify with the tester before finishing\n"16 },17 "judge": {18 "image_tag": "eb22343ce84e",19 "eval_cmd": "cd /home/workspace/vibrating_path && python3 /tmp/eval_vibrating_path.py",20 "eval_timeout": 600,21 "parser": "score_sum",22 "score_direction": "minimize",23 "selection": "score_first",24 "rescale": {25 "kind": "piecewise_log_min",26 "baseline": 31051708.0,27 "rank30": 3288665.0,28 "rank1": 2181554.0,29 "super_anchor": 1904776.2530 }31 }32}33