SHYAMSATHISH005/data-cleaning-openenv

Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference

Data Cleaning OpenEnv

A production-grade OpenEnv environment for training and evaluating data-cleaning agents on realistic ETL-style tasks. Agents interact through a structured reset / step / state loop, applying discrete repair operations to corrupted dataframes and receiving dense reward signals tied to ground-truth accuracy.

Why This Environment

Data cleaning is one of the highest-cost, highest-frequency tasks in real data engineering. It is also one of the least studied in agent evaluation — most benchmarks focus on code generation or web navigation. This environment fills that gap.

The three tasks model corruptions that appear in actual production pipelines: type coercion failures, inconsistent date and phone formatting, missing values, outlier constraint violations, and duplicate records. An agent that scores well here is solving a problem that data teams face daily.

Task Suite

Each task uses a seeded synthetic dataset so results are fully reproducible across runs.

Observation Space

Every step and reset call returns an Observation with the following fields:

Action Space

All actions follow a single JSON envelope:

{
  "name": "<action_name>",
  "params": { }
}

Reward Design

The reward function provides dense signal across the full trajectory, not just at episode end.

step_reward  = clamp(accuracy_delta, 0.0, 1.0)
invalid_penalty = -0.05   (applied on malformed or no-op actions)
submit_bonus = +1.0       (applied only when accuracy == 1.0 at submit)

Agents that improve accuracy at every step receive consistent positive feedback. Agents that repeat ineffective actions are penalised. The submit bonus incentivises committing once the dataframe is genuinely clean rather than running out the step budget.

API Reference

Base URL (local): http://127.0.0.1:7860

Reset

curl -s -X POST http://127.0.0.1:7860/reset \
  -H "Content-Type: application/json" \
  -d '{"task_id": "fix_types"}'

Step

curl -s -X POST http://127.0.0.1:7860/step \
  -H "Content-Type: application/json" \
  -d '{"name": "cast_type", "params": {"column": "amount", "dtype": "float"}}'

Submit

curl -s -X POST http://127.0.0.1:7860/step \
  -H "Content-Type: application/json" \
  -d '{"name": "submit", "params": {}}'

Setup

Run Locally

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
uvicorn app:app --host 0.0.0.0 --port 7860

Run with Docker

docker build -t data-cleaning-openenv .
docker run --rm -p 7860:7860 data-cleaning-openenv

Baseline Inference

The inference.py script runs a deterministic baseline plan against all three tasks, then falls back to an LLM agent for any remaining steps. It uses the OpenAI client and reads credentials from environment variables.

export ENV_BASE_URL=http://127.0.0.1:7860
export API_BASE_URL=https://your-openai-compatible-endpoint/v1
export MODEL_NAME=gpt-4o-mini
export HF_TOKEN=your_token_here
python inference.py

Expected output format:

[START] task_id=fix_types model=gpt-4o-mini ...
[STEP]  task_id=fix_types step=1 action={"name":"cast_type",...} reward=0.500000 score=0.500000 done=false
[STEP]  task_id=fix_types step=2 action={"name":"submit",...}    reward=0.000000 score=0.500000 done=true
[END]   task_id=fix_types steps=2 final_score=0.500000 status=max_steps
...
[END]   run_summary tasks=3 average_score=0.xxxxxx

Validation

Run the pre-submission validation script to check all endpoints respond correctly before deploying:

python pre_validation.py

To skip Docker checks during local iteration:

SKIP_DOCKER=1 python pre_validation.py

Hugging Face Deployment

This repository is configured as a Docker Space. To deploy:

1. 1.Create a Space at huggingface.co/spaces with the Docker SDK.
2. 2.Add this repository as the Space remote and push.
3. 3.The container starts on port 7860 automatically — no extra configuration needed.

Set the following Space secrets for inference runs:

Project Structure

data-cleaning-openenv/
├── app.py              FastAPI server — reset / step / state endpoints
├── env.py              Core environment logic and action executors
├── models.py           Pydantic models for Observation, Action, Reward
├── inference.py        Baseline inference script (OpenAI client)
├── pre_validation.py   Pre-submission validation checks
├── openenv.yaml        OpenEnv spec metadata
├── requirements.txt
├── Dockerfile
└── README.md

Environment Metadata