SHYAMSATHISH005/data-cleaning-openenv

1---2title: Data Cleaning OpenEnv3emoji: 🧹4colorFrom: blue5colorTo: green6sdk: docker7pinned: false8---9 10Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference11 12# Data Cleaning OpenEnv13 14A production-grade OpenEnv environment for training and evaluating data-cleaning agents on realistic ETL-style tasks. Agents interact through a structured `reset / step / state` loop, applying discrete repair operations to corrupted dataframes and receiving dense reward signals tied to ground-truth accuracy.15 16---17 18## Why This Environment19 20Data cleaning is one of the highest-cost, highest-frequency tasks in real data engineering. It is also one of the least studied in agent evaluation — most benchmarks focus on code generation or web navigation. This environment fills that gap.21 22The three tasks model corruptions that appear in actual production pipelines: type coercion failures, inconsistent date and phone formatting, missing values, outlier constraint violations, and duplicate records. An agent that scores well here is solving a problem that data teams face daily.23 24---25 26## Task Suite27 28| ID | Difficulty | Description |29|----|------------|-------------|30| `fix_types` | Easy | All columns are stored as strings. Cast each to its correct type: integer, float, boolean, datetime. |31| `normalize_dedupe` | Medium | Names, dates, and phone numbers are in inconsistent formats. Approximately 15% of rows are duplicates. Normalize and deduplicate. |32| `full_pipeline` | Hard | A heavily corrupted dataset with wrong types, missing values, age and revenue constraint violations, future signup dates, and duplicate rows. Repair everything. |33 34Each task uses a seeded synthetic dataset so results are fully reproducible across runs.35 36---37 38## Observation Space39 40Every `step` and `reset` call returns an `Observation` with the following fields:41 42| Field | Type | Description |43|-------|------|-------------|44| `dataframe_preview` | `list[dict]` | First 5 rows as a list of records |45| `markdown_preview` | `string` | Same rows rendered as a markdown table |46| `null_counts` | `dict[str, int]` | Per-column null value counts |47| `validation_errors` | `list[string]` | Human-readable list of remaining issues |48| `accuracy` | `float` | Current score against ground truth (0.0–1.0) |49| `step_count` | `int` | Number of actions taken so far |50 51---52 53## Action Space54 55All actions follow a single JSON envelope:56 57```json58{59  "name": "<action_name>",60  "params": { }61}62```63 64| Action | Key Params | Effect |65|--------|-----------|--------|66| `cast_type` | `column`, `dtype` | Cast a column to `float`, `int`, `str`, or `datetime` |67| `fill_missing` | `column`, `strategy` | Fill nulls via `mean`, `median`, `mode`, `ffill`, `bfill`, or a literal value |68| `drop_duplicates` | `subset` (optional) | Remove duplicate rows, optionally scoped to a column subset |69| `normalize_dates` | `column` | Parse and reformat dates to ISO-8601 (`YYYY-MM-DD`) |70| `replace` | `column`, `old_value`, `new_value` | Replace a specific value in a column |71| `clamp_outliers` | `column`, `low`, `high` | Clip numeric values to a valid range |72| `submit` | — | Finalise the episode and record the score |73 74---75 76## Reward Design77 78The reward function provides dense signal across the full trajectory, not just at episode end.79 80```81step_reward  = clamp(accuracy_delta, 0.0, 1.0)82invalid_penalty = -0.05   (applied on malformed or no-op actions)83submit_bonus = +1.0       (applied only when accuracy == 1.0 at submit)84```85 86Agents that improve accuracy at every step receive consistent positive feedback. Agents that repeat ineffective actions are penalised. The submit bonus incentivises committing once the dataframe is genuinely clean rather than running out the step budget.87 88---89 90## API Reference91 92Base URL (local): `http://127.0.0.1:7860`93 94| Method | Endpoint | Description |95|--------|----------|-------------|96| `GET` | `/` | Service metadata and version |97| `GET` | `/health` | Liveness check — returns `{"ok": true}` |98| `POST` | `/reset` | Start a new episode for a given task |99| `POST` | `/step` | Apply one cleaning action |100| `GET` | `/state` | Return the current observation without advancing the episode |101 102### Reset103 104```bash105curl -s -X POST http://127.0.0.1:7860/reset \106  -H "Content-Type: application/json" \107  -d '{"task_id": "fix_types"}'108```109 110### Step111 112```bash113curl -s -X POST http://127.0.0.1:7860/step \114  -H "Content-Type: application/json" \115  -d '{"name": "cast_type", "params": {"column": "amount", "dtype": "float"}}'116```117 118### Submit119 120```bash121curl -s -X POST http://127.0.0.1:7860/step \122  -H "Content-Type: application/json" \123  -d '{"name": "submit", "params": {}}'124```125 126---127 128## Setup129 130### Run Locally131 132```bash133python3 -m venv .venv134source .venv/bin/activate135pip install -r requirements.txt136uvicorn app:app --host 0.0.0.0 --port 7860137```138 139### Run with Docker140 141```bash142docker build -t data-cleaning-openenv .143docker run --rm -p 7860:7860 data-cleaning-openenv144```145 146---147 148## Baseline Inference149 150The `inference.py` script runs a deterministic baseline plan against all three tasks, then falls back to an LLM agent for any remaining steps. It uses the OpenAI client and reads credentials from environment variables.151 152```bash153export ENV_BASE_URL=http://127.0.0.1:7860154export API_BASE_URL=https://your-openai-compatible-endpoint/v1155export MODEL_NAME=gpt-4o-mini156export HF_TOKEN=your_token_here157python inference.py158```159 160Expected output format:161 162```163[START] task_id=fix_types model=gpt-4o-mini ...164[STEP]  task_id=fix_types step=1 action={"name":"cast_type",...} reward=0.500000 score=0.500000 done=false165[STEP]  task_id=fix_types step=2 action={"name":"submit",...}    reward=0.000000 score=0.500000 done=true166[END]   task_id=fix_types steps=2 final_score=0.500000 status=max_steps167...168[END]   run_summary tasks=3 average_score=0.xxxxxx169```170 171---172 173## Validation174 175Run the pre-submission validation script to check all endpoints respond correctly before deploying:176 177```bash178python pre_validation.py179```180 181To skip Docker checks during local iteration:182 183```bash184SKIP_DOCKER=1 python pre_validation.py185```186 187---188 189## Hugging Face Deployment190 191This repository is configured as a Docker Space. To deploy:192 1931. Create a Space at `huggingface.co/spaces` with the **Docker** SDK.1942. Add this repository as the Space remote and push.1953. The container starts on port `7860` automatically — no extra configuration needed.196 197Set the following Space secrets for inference runs:198 199| Variable | Purpose |200|----------|---------|201| `API_BASE_URL` | OpenAI-compatible LLM endpoint |202| `MODEL_NAME` | Model identifier for inference |203| `HF_TOKEN` | API key / Hugging Face token |204| `ENV_BASE_URL` | Override if environment is not on localhost |205 206---207 208## Project Structure209 210```211data-cleaning-openenv/212├── app.py              FastAPI server — reset / step / state endpoints213├── env.py              Core environment logic and action executors214├── models.py           Pydantic models for Observation, Action, Reward215├── inference.py        Baseline inference script (OpenAI client)216├── pre_validation.py   Pre-submission validation checks217├── openenv.yaml        OpenEnv spec metadata218├── requirements.txt219├── Dockerfile220└── README.md221```222 223---224 225## Environment Metadata226 227| Property | Value |228|----------|-------|229| Interface | OpenEnv v1 (`reset / step / state`) |230| Reward | Dense, per-step accuracy delta |231| Episodes | Seeded, reproducible |232| Max steps | 15 / 20 / 30 (easy / medium / hard) |233| Serving port | 7860 |234| Runtime | Python 3.10, FastAPI, pandas, numpy |