KurtDu/SpaceCode-Bench
SpaceCode-Bench Construct, reconstruct and edit native 3D environments with Blender code. Website · Code and evaluator · Protocol · Complete worked example SpaceCode-Bench is a procedural development benchmark with 1,200 materialized scenes, 3,600 task bundles and 4,800 execution conditions. Each task publishes a complete numerical contract. Evaluation reopens the native .blend, extracts geometry and controlled views, and measures the declared requirements. Construction starts… See the full description on the dataset page: https://huggingface.co/datasets/KurtDu/SpaceCode-Bench.
0187
1# Worked example: edit a reading library2 3Task: `reading_library__open_hall__00__edit`. This is a repository-owned author control, not a model submission.4 5The complete public request is in `task.json`, `prompt.txt` and `contract.json`. The solver receives `starter.blend` and the published edit region. `edited.blend` is the authored positive control; `base.blend` supplies the original reference environment. These native files have the same SHA256 bindings as the website's corresponding controls.6 7| Control | A | G | F | P | Native score | Critical pass |8| --- | ---: | ---: | ---: | ---: | ---: | --- |9| Requested edit | 1 | 1 | 1 | 1 | 100 | true |10| Unchanged starter | 1 | 0.9787234043 | 1 | 1 | 49 | false |11 12The unchanged starter preserves the protected environment but fails the requested intervention. A high mean geometric score cannot override that critical failure. R is inapplicable to editing; C, V and full_score are unmeasured.13 14`evidence/` and `unchanged-starter-evidence/` contain actual independently extracted geometry, inspection, fixed renders, bound preservation evidence and clause-level results. `starter-evidence/` is the preservation reference. `scoring-requested_edit/` and `scoring-unchanged_starter/` record evaluator-7 numerical recomputation, its process limits and actual dependency identities. `provenance.json` identifies the native artifacts and distinguishes this new numerical run from the preserved extraction. No model API was called.15 16To rebuild the authored native scenes with Blender 4.2.23:17 18```bash19blender --background --factory-startup --python build_scene.py -- \20 --repository /absolute/path/to/SpaceCode-Bench --out /absolute/path/to/rebuilt21```22 23The builder uses `spec.json` and the repository's procedural mesher. A rebuilt `.blend` may differ byte-for-byte because of save metadata; compare extracted geometry under the same documented toolchain. Author generation is a reproducibility aid and is not an allowed solver input.24 25For model evaluation, download the complete dataset and use the repository's `spacecode batch` or `spacecode evaluate` entry. They validate public/private resources and isolate candidate execution before extraction and numerical scoring. The standalone legacy `spacecode score` command does not implement this native protocol.26 27Data: CC BY 4.0, Yuhao Du. Builder code: MIT. See the licenses at the repository root.28