emrekuruu/spreadsheetbench-openenv

SpreadsheetBench OpenEnv

A standalone, typed OpenEnv package for the SpreadsheetBench v0.1 release. It mirrors the SkillOpt codegen harness in its default multi-turn mode: reset returns one task prompt, the client submits one complete Python script per turn, the server executes it on the first hidden case and returns any execution error as feedback for up to 30 turns within a 600 s task budget, then runs the final script independently across every hidden case, grades the outputs with official cell-value semantics, and ends the episode. Reset requires training_feedback: true reproduces SkillOpt's training-only gold check (a wrong first-case output is fed back as a cell-level report with expected values hidden), false is the evaluation protocol in which no grading evidence reaches the agent.

The canonical package catalog contains 901 validated instruction-bearing tasks split into 700 development and 201 frozen test tasks with source-group isolation. See src/spreadsheetbench_env/assets/tasks/exclusions.json for the explicit eleven-task exclusion report.

Local server

Install with the server extra, set SPREADSHEETBENCH_DATA to a runtime root containing the canonical spreadsheet/<task-id>/ workbook directories, and run server.

Generated scripts execute in bounded subprocesses with scrubbed environments, resource limits, process-group cleanup, and private per-session directories. The Docker deployment leases a distinct unprivileged UID to every active session while the hidden runtime tree remains root-only. This is pragmatic process containment, not a complete network or kernel sandbox.