Datasetpublic

ByteDance-Seed/EdgeBench

Overview EdgeBench is a benchmark of 134 real-world tasks for evaluating how autonomous AI agents learn from real-world environments. Instead of measuring one-shot performance, EdgeBench places agents in executable task environments with realistic, multi-level feedback and lets them iterate for 12+ hours per task — tracking the full trajectory of improvement, not just the final score. We publicly release 51 tasks… See the full description on the dataset page: https://huggingface.co/datasets/ByteDance-Seed/EdgeBench.

sourceHugging Facecc-by-4.0updated 2mo agoView on Hugging Face

84likes7.5kdownloads

rootgraph_node_classification.json

graph_node_classification.json36 linesDownload Raw Back to root

1{2  "task_id": "graph_node_classification",3  "name": "Graph Node Classification",4  "category": "Scientific Problems & ML",5  "base_image": "python",6  "platform": "linux/amd64",7  "internet": false,8  "cwd": "/home/workspace/045_graph_node_classification",9  "submit_paths": [10    "solution/solution.py"11  ],12  "submit_exclude": [13    "tests/",14    "**/__pycache__/**",15    "**/*.pyc"16  ],17  "work": {18    "image_tag": "4d2c1f502d14",19    "specs_dir": "/home/workspace/045_graph_node_classification",20    "agent_query": "Solve the graph node classification task described in `README.md` and `solution/problem.md`."21  },22  "judge": {23    "image_tag": "f2ddd52992ae",24    "eval_cmd": "cd /home/workspace/045_graph_node_classification && CUDA_VISIBLE_DEVICES= NVIDIA_VISIBLE_DEVICES=none python judge_eval.py",25    "eval_timeout": 1800,26    "parser": "structured_json",27    "score_direction": "maximize",28    "selection": "valid_then_score",29    "rescale": {30      "kind": "linear",31      "lower": 0.0,32      "upper": 1.033    }34  }35}36