KingHero121/dataforge-env

1---2title: DataForge Env3emoji: 🤖4colorFrom: blue5colorTo: green6sdk: docker7app_file: server/app.py8pinned: false9---10 11# DataForge-Env 🔧12 13> A production-grade OpenEnv environment for evaluating LLM-based agents on real-world data cleaning, validation, and multi-table reconciliation tasks.14 15---16 17## Overview18 19DataForge-Env wraps a stateful data-cleaning sandbox. Agents interact via structured actions to fix dirty datasets — filling nulls, casting types, normalising values, joining tables, and satisfying business rules.20 21Rewards are **dense** and **deterministic**: agents receive granular feedback after every step, enabling RL-style training and scientific benchmarking.22 23## Tasks24 25| ID | Name | Difficulty | Max Steps | Description |26|----|------|-----------|-----------|-------------|27| `easy` | The Untidy Retailer | Easy | 15 | Fill missing emails, remove duplicates, trim whitespace |28| `medium` | Financial Anomaly | Medium | 20 | Parse currency strings, unify dates, cap outliers |29| `hard` | Supply Chain Reconciliation | Hard | 25 | Normalise SKU keys, join tables, compute inventory value |30 31## Quick Start32 33### Local34```bash35pip install -r requirements.txt36python -m server.app37# Server runs on http://localhost:786038```39 40### Docker41```bash42docker build -t dataforge-env .43docker run -p 7860:7860 dataforge-env44```45 46### API Usage47 48**Reset** (start an episode):49```bash50curl -X POST http://localhost:7860/reset \51  -H "Content-Type: application/json" \52  -d '{"task_id": "easy"}'53```54 55**Step** (apply an action):56```bash57curl -X POST http://localhost:7860/step \58  -H "Content-Type: application/json" \59  -d '{"action": {"action_type": "fill_missing", "params": {"column": "email", "strategy": "constant", "fill_value": "unknown@example.com"}}}'60```61 62## Action Space63 64| Action | Key Params |65|--------|-----------|66| `fill_missing` | `column`, `strategy` (mean/median/mode/constant/drop), `fill_value` |67| `drop_duplicates` | `subset` (optional list of columns) |68| `cast_type` | `column`, `target_dtype` (int/float/str/datetime) |69| `normalize` | `column`, `method` (trim/lower/upper/strip_currency/unify_date/strip_prefix/map_values/clip) |70| `join` | `right_table`, `left_on`, `right_on`, `how` |71| `validate` | (no params — returns current validation errors) |72 73## Reward Formula74 75```76R = 0.3 × C_schema + 0.2 × C_nulls + 0.1 × C_dupes + 0.4 × C_logic − 0.01 × step_penalty77```78 79All components and the final reward are normalised to **[0, 1]**.80 81## Inference Script82 83```bash84export API_BASE_URL=https://api-inference.huggingface.co/v185export MODEL_NAME=meta-llama/Llama-3-70B-Instruct86export HF_TOKEN=hf_...87export ENV_URL=http://localhost:786088export TASK_ID=easy89 90python inference.py91```92 93Output follows strict `[START]` / `[STEP]` / `[END]` format.94 95## Project Structure96 97```98├── openenv.yaml          # Environment specification99├── env/100│   ├── models.py         # Pydantic schemas101│   ├── env.py            # Core environment class102│   ├── tasks.py          # Task definitions & data generators103│   └── graders.py        # Deterministic grading104├── server/105│   └── app.py            # FastAPI server106├── inference.py          # LLM agent inference script107├── Dockerfile            # Container definition108├── requirements.txt      # Python dependencies109└── README.md             # This file110```111 112## License113 114MIT115