Datasetpublic

ByteDance-Seed/EdgeBench

Overview EdgeBench is a benchmark of 134 real-world tasks for evaluating how autonomous AI agents learn from real-world environments. Instead of measuring one-shot performance, EdgeBench places agents in executable task environments with realistic, multi-level feedback and lets them iterate for 12+ hours per task — tracking the full trajectory of improvement, not just the final score. We publicly release 51 tasks… See the full description on the dataset page: https://huggingface.co/datasets/ByteDance-Seed/EdgeBench.

sourceHugging Facecc-by-4.0updated 2mo agoView on Hugging Face

84likes7.1kdownloads

rootinteger_compression_codec.json

integer_compression_codec.json39 linesDownload Raw Back to root

1{2  "task_id": "integer_compression_codec",3  "name": "Integer Compression Codec",4  "category": "Systems & Software Engineering",5  "base_image": "cpp",6  "platform": "linux/amd64",7  "internet": false,8  "cwd": "/home/workspace/integer-compression-codec",9  "submit_paths": [10    "include/",11    "src/",12    "Makefile"13  ],14  "submit_exclude": [15    "**/__pycache__/",16    "outputs/",17    "data/",18    ".judge-bin/"19  ],20  "work": {21    "image_tag": "fa3895856662",22    "specs_dir": "/home/workspace/integer-compression-codec",23    "agent_query": "## Role\\n\\nYou are an expert systems and performance engineer improving a uint32 integer compression codec.\\n\\n## Task\\n\\nImplement a stronger `libintcompress.so` by editing the C++ code in `src/` while preserving the ABI in `include/intcompress.h`. Your goal is to improve both compression ratio and decode throughput on the provided datasets.\\n\\n## Files\\n\\n- `problem.md`: task description and constraints\\n- `task_input.md`: input / output contract\\n- `scoring_scheme.md`: public scoring rules\\n- `soft_environment.md`: environment description\\n- `include/intcompress.h`: fixed ABI\\n- `src/codec.cpp`: baseline implementation\\n- `tools/verify.cpp`: local correctness check\\n- `tools/bench.cpp`: local benchmarking tool\\n\\n## Working Strategy\\n\\n1. Read the docs first.\\n2. Keep `include/intcompress.h` unchanged.\\n3. Improve `src/` with better block encoding, delta transforms, bit-packing, SIMD, or adaptive strategies.\\n4. Use `make`, `./tools/verify`, and `./tools/bench --all ./libintcompress.so` for local iteration.\\n5. Submit intermediate versions frequently.\\n\\n## Rules\\n\\n- Do not use network access, subprocess tricks, or external services.\\n- Do not depend on third-party compression libraries.\\n- Keep the exported `ic_*` symbols intact.\\n- Correctness matters: `decode(encode(X))` must exactly recover `X`."24  },25  "judge": {26    "image_tag": "27063396e02a",27    "eval_cmd": "cd /home/workspace/integer-compression-codec && python3 runner_judge.py",28    "eval_timeout": 2400,29    "parser": "structured_json",30    "score_direction": "maximize",31    "selection": "score_first",32    "rescale": {33      "kind": "linear",34      "lower": 0.0,35      "upper": 100.036    }37  }38}39