CoolFace
Apppublic

Adarsh2004/real_world_task_sim

sourceHugging Faceupdated 5mo agoView on Hugging Face
0likes
App README

Real-World Task Simulation Environment (OpenEnv)

A standardized, containerized environment for evaluating AI agents on real-world tasks. This environment implements the OpenEnv specification and provides three tasks with incremental difficulty.

Tasks & Difficulty

  1. 1.Email Triage (Easy): Classify a set of emails into categories: Work, Personal, or Spam.
  2. 2.Data Cleaning (Medium): Clean a CSV file by normalizing date formats, removing duplicates, and handling missing values.
  3. 3.Code Review (Hard): Identify and fix a common Python bug (mutable default argument) in a code snippet.

Action & Observation Spaces

Action Space (RealWorldTaskAction)

  • —task_id: The ID of the task being addressed.
  • —action_type: The operation being performed (e.g., "submit").
  • —payload: A dictionary containing task-specific results (e.g., { "predictions": { "1": "Work" } }).

Observation Space (RealWorldTaskObservation)

  • —current_task: The ID of the active task.
  • —prompt: Instructions for the agent.
  • —data: The raw data for the task (JSON list, CSV string, or Python code).
  • —done: Boolean indicating if all tasks are complete.
  • —reward: Cumulative score (0.0 to 1.0 per task).
  • —last_reward: Detailed feedback and score for the previous action.

Reward Function

  • —Incremental: Agents receive rewards at each step (one step per task).
  • —Programmatic Graders: Success is measured by deterministic logic (regex for dates, exact match for categories, code analysis for fixes).
  • —Penalties: Incorrect task IDs or invalid payloads result in zero reward for that step.

Usage

Local Development

  1. 1.Install dependencies:
bash
   pip install openenv-core pandas openai
   uv lock
  1. 1.Run the environment:
bash
   python -m server.app

Baseline Inference

Run the baseline script to evaluate an agent (requires OPENAI_API_KEY or HF_TOKEN):

bash
python inference.py

Deployment

Deploy to Hugging Face Spaces using the OpenEnv CLI:

bash
openenv push

Specification Compliance

Validated using openenv validate.

  • —Typed Pydantic models.
  • —Standard reset(), step(), state() interface.
  • —openenv.yaml manifest.
  • —Containerized execution via Dockerfile.