CoolFace
Apppublic

c-varun/metaXscalar

sourceHugging Faceupdated 6mo agoView on Hugging Face
0likes
Meta Hackathon Problem Statement.md79 linesDownload Raw Back to root
1## **Functional Requirements**2 3### **1\. Real-World Task Simulation**4 5The environment must represent tasks that humans actually perform in real settings—no games or toy problems.6 7Examples include email triage, code review, data cleaning, scheduling, customer support, and content moderation.8 9---10 11### **2\. OpenEnv Specification Compliance**12 13The environment must fully implement the OpenEnv interface, including:14 15* Typed **Observation**, **Action**, and **Reward** models using Pydantic  16* step(action) → returns (observation, reward, done, info)  17* reset() → returns the initial observation  18* state() → returns the current state  19* An openenv.yaml file containing metadata20 21The implementation must successfully pass validation via openenv validate.22 23---24 25### **3\. Minimum of Three Tasks with Agent Graders**26 27* Provide at least **three tasks**, each with a clearly defined objective  28* Tasks should span increasing difficulty: **easy → medium → hard**  29* Each task must include a **programmatic grader** that assigns a score between **0.0 and 1.0**  30* Grading criteria must be **clear, deterministic, and reproducible**31 32---33 34### **4\. Meaningful Reward Function**35 36* The reward function must provide feedback throughout the task trajectory, not just at completion  37* It should reward **incremental progress** toward the objective  38* It must penalize undesirable behaviors such as infinite loops or destructive actions39 40---41 42### **5\. Baseline Inference Script**43 44* Include an inference script that uses the OpenAI API client to evaluate a model within the environment  45* API credentials must be read from environment variables (HF\_TOKEN)  46* The script should produce a **reproducible baseline score** across all tasks47 48---49 50## **Non-Functional Requirements**51 52### **1\. Deployment on Hugging Face Spaces**53 54* The environment must be deployable as a containerized Hugging Face Space  55* It should be tagged with openenv56 57---58 59### **2\. Containerized Execution**60 61* Provide a working **Dockerfile**  62* The environment must build and run successfully using:  63  * docker build  64  * docker run65 66---67 68### **3\. Documentation**69 70The README must include:71 72* Environment overview and motivation  73* Definitions of action and observation spaces  74* Task descriptions with expected difficulty levels  75* Setup and usage instructions  76* Baseline performance scores77 78Additional Guideline: [Meta OpenEnv Hackathon: Guidelines]79