RohanExploit/Meta-hackathon

1The Task2 3Build a complete, real-world OpenEnv environment that an AI agent can learn from through the standard  step() / reset() / state()  API.4 5REBUILT SPECIFICATION (v2.0):6============================7Domain: Dynamic Multi-Channel Retail with Disruption Recovery8 9Real-world scenario: A retailer managing inventory across 3 product categories10with multiple customer segments (luxury/budget), dynamic demand shocks, and11supply chain disruptions. Agent must adapt pricing, allocate inventory strategically,12and recover from disruptions (stockouts, supplier delays, demand collapse).13 14This is genuinely challenging: retailers face exactly this complexity daily.15It's novel: no existing OpenEnv environment models multi-segment pricing + disruptions.16 17Key Requirements at a Glance18 19Must simulate a real-world task (not games or toys)20 21Implement full OpenEnv spec: typed models, step()/reset()/state(), openenv.yaml22 23Minimum 3 tasks with agent graders (easy → medium → hard, scores 0.0–1.0)24 25Meaningful reward function with partial progress signals26 27Baseline inference script with reproducible scores28 29Deploy to Hugging Face Spaces + working Dockerfile30 31README with environment description, action/observation spaces, setup instructions32 33Functional Requirements34 35Real-world task simulation36 37The environment must simulate a task humans actually do. Not games, not toys. Examples: email triage, code review, data cleaning, scheduling, customer support, content moderation.38 39OpenEnv spec compliance40 41Implement the full OpenEnv interface: typed Observation, Action, and Reward Pydantic models. step(action) → returns observation, reward, done, info. reset() → returns initial observation. state() → returns current state. openenv.yaml with metadata. Tested via openenv validate.42 43Minimum 3 tasks with agent graders44 45Each task defines a concrete objective an agent must accomplish, with a programmatic grader that scores performance (0.0–1.0). Tasks should range: easy → medium → hard. Graders must have clear, deterministic success/failure criteria.46 47Meaningful reward function48 49Provides signal over the full trajectory (not just binary end-of-episode). Rewards partial progress toward task completion. Penalizes clearly undesirable behavior (e.g. infinite loops, destructive actions).50 51Baseline inference script52 53Uses the OpenAI API client to run a model against the environment. Reads API credentials from environment variables (OPENAI_API_KEY). Produces a reproducible baseline score on all 3 tasks.54 55Detailed Requirements56 57Non-Functional Requirements58 59Deploys to a Hugging Face Space60 61Environment must run as a containerized HF Space tagged with openenv.62 63Containerized execution64 65Must include a working Dockerfile. The environment should start cleanly with docker build + docker run.66 67Documentation68 69README must include: environment description and motivation, action and observation space definitions, task descriptions with expected difficulty, setup and usage instructions, baseline scores.70 71Parameter72 73Weight74 75Description76 77Real-world utility78 7930%80 81Does the environment model a genuine task? Would someone actually use this to train or evaluate agents?82 83Task & grader quality84 8525%86 87Are tasks well-defined with clear objectives? Do graders accurately and fairly measure success? Meaningful difficulty progression?88 89Environment design90 9120%92 93Clean state management, sensible action/observation spaces, good reward shaping, proper episode boundaries.94 95Code quality & spec compliance96 9715%98 99Follows OpenEnv spec, clean project structure, typed models, documented, tested, Dockerfile works.100 101Creativity & novelty102 10310%104 105Novel problem domain, interesting mechanics, clever reward design, original approach.106 107Scoring Breakdown108 109Real-world utility (30%)110 111•  0–5: Toy/artificial problem with no practical application112 113•  6–15: Valid domain but shallow modeling of the real task114 115•  16–25: Good domain modeling, would be useful for agent evaluation116 117•  26–30: Excellent — fills a real gap, immediate value for the RL/agent community118 119Task & grader quality (25%)120 121•  3+ tasks with difficulty range?122 123•  Graders produce scores between 0.0–1.0?124 125•  Graders deterministic and reproducible?126 127•  Hard task genuinely challenges frontier models?128 129Environment design (20%)130 131•  reset() produces clean state?132 133•  Action/observation types well-designed and documented?134 135•  Reward function provides useful varying signal (not just sparse)?136 137•  Episode boundaries sensible?138 139Code quality & spec compliance (15%)140 141•  openenv validate passes?142 143•  docker build && docker run works?144 145•  HF Space deploys and responds?146 147•  Baseline script runs and reproduces scores?148 149Creativity & novelty (10%)150 151•  Domain we haven’t seen in OpenEnv before?152 153•  Reward design has interesting properties?154 155•  Clever mechanics that make the environment engaging?156 157Evaluation Criteria158 159Phase 1: Automated Validation160 161Pass/fail gate — HF Space deploys, OpenEnv spec compliance, Dockerfile builds, baseline reproduces, 3+ tasks with graders.162 163Phase 2: Agentic Evaluation164 165Scored — baseline agent re-run, standard Open LLM agent (e.g. Nemotron 3 Super) run against all environments, score variance check.166 167Phase 3: Human Review168 169Top submissions reviewed by Meta and Hugging Face engineers for real-world utility, creativity, and exploit checks.170 171Disqualification Criteria172 173Environment does not deploy or respond174 175Plagiarized or trivially modified existing environments176 177Graders that always return the same score178 179No baseline inference script180 181How Judging works182 183Pre-Submission Checklist  — all must pass or you're disqualified184 185HF Space deploys186 187Automated ping to the Space URL — must return 200 and respond to reset()188 189OpenEnv spec compliance190 191Validate openenv.yaml, typed models, step()/reset()/state() endpoints192 193Dockerfile builds194 195Automated docker build on the submitted repo196 197Baseline reproduces198 199Run the submitted inference script — must complete without error and produce scores200 2013+ tasks with graders202 203Enumerate tasks, run each grader, verify scores in 0.0–1.0 range204 205Additional Instructions206 207Before submitting, ensure the following variables are defined in your environment configuration:  208 209API_BASE_URL   The API endpoint for the LLM.  210 211MODEL_NAME     The model identifier to use for inference. 212 213HF_TOKEN       Your Hugging Face / API key.214 215The inference script must be named `inference.py` and placed in the root directory of the project216 217Participants must use OpenAI Client for all LLM calls using above variables218 219Infra Restrictions220 221Runtime of inference script should be less than 20min 222 223Make sure your env and inference can run on a machine with vcpu=2, memory=8gb224 225"""226Inference Script Example227===================================228MANDATORY229- Before submitting, ensure the following variables are defined in your environment configuration:230    API_BASE_URL   The API endpoint for the LLM.231    MODEL_NAME     The model identifier to use for inference.232    HF_TOKEN       Your Hugging Face / API key.233    234- The inference script must be named `inference.py` and placed in the root directory of the project235- Participants must use OpenAI Client for all LLM calls using above variables236"""237 238import os239import re240import base64241import textwrap242from io import BytesIO243from typing import List, Optional, Dict244 245from openai import OpenAI246import numpy as np247from PIL import Image248 249from browsergym_env import BrowserGymAction, BrowserGymEnv250 251API_BASE_URL = os.getenv("API_BASE_URL") // "https://router.huggingface.co/v1"252API_KEY = os.getenv("HF_TOKEN") or os.getenv("API_KEY")253MODEL_NAME = os.getenv("MODEL_NAME")254MAX_STEPS = 8255MAX_DOM_CHARS = 3500256TEMPERATURE = 0.2257MAX_TOKENS = 200258FALLBACK_ACTION = "noop()"259 260DEBUG = True261ACTION_PREFIX_RE = re.compile(262    r"^(action|next action)\s*[:\-]\s*",263    re.IGNORECASE,264)265ACTION_PATTERN = re.compile(r"[A-Za-z_]+\s*\(.*\)", re.DOTALL)266 267 268SYSTEM_PROMPT = textwrap.dedent(269    """270    You control a web browser through BrowserGym.271    Reply with exactly one action string.272    The action must be a valid BrowserGym command such as:273    - noop()274    - click('<BID>')275    - type('selector', 'text to enter')276    - fill('selector', 'text to enter')277    - send_keys('Enter')278    - scroll('down')279    Use single quotes around string arguments.280    When clicking, use the BrowserGym element IDs (BIDs) listed in the user message.281    If you are unsure, respond with noop().282    Do not include explanations or additional text.283    """284).strip()285 286 287def build_history_lines(history: List[str]) -> str:288    if not history:289        return "None"290    return "\n".join(history[-4:])291 292 293def extract_screenshot_uri(observation) -> Optional[str]:294    if observation.screenshot is None:295        return None296    screen_array = np.array(observation.screenshot, dtype=np.uint8)297    image = Image.fromarray(screen_array)298    buffer = BytesIO()299    image.save(buffer, format="PNG")300    buffer.seek(0)301    data_uri = base64.b64encode(buffer.read()).decode("utf-8")302    return f"data:image/png;base64,{data_uri}"303 304 305def extract_clickable_elements(observation) -> List[Dict[str, str]]:306    """Collect BrowserGym element IDs that can be clicked."""307 308    metadata = getattr(observation, "metadata", {}) or {}309    obs_dict = metadata.get("browsergym_obs", {}) or {}310    extra_props = obs_dict.get("extra_element_properties", {}) or {}311 312    clickables: List[Dict[str, str]] = []313    for bid, props in extra_props.items():314        if not props.get("clickable"):315            continue316 317        bbox = props.get("bbox") or []318        bbox_str = ", ".join(bbox) if bbox else "?"319        clickables.append(320            {321                "bid": str(bid),322                "bbox": bbox_str,323            }324        )325 326    # Keep a stable ordering for readability327    clickables.sort(key=lambda item: item["bid"])328    return clickables329 330 331def build_user_prompt(step: int, observation, history: List[str]) -> str:332    goal = observation.goal or "(not provided)"333    url = observation.url or "(unknown)"334    error_note = "Yes" if observation.last_action_error else "No"335 336    clickables = extract_clickable_elements(observation)337    if clickables:338        actions_hint = "\n".join(339            f"    - {item['bid']} (bbox: {item['bbox']})" for item in clickables340        )341    else:342        actions_hint = "    (none detected)"343 344    prompt = textwrap.dedent(345        f"""346        Step: {step}347        Goal: {goal}348        Current URL: {url}349        Previous steps:350        {build_history_lines(history)}351        Last action error: {error_note}352        Available clickable element IDs: {actions_hint}353        Reply with exactly one BrowserGym action string.354        """355    ).strip()356    return prompt357 358 359def parse_model_action(response_text: str) -> str:360    if not response_text:361        return FALLBACK_ACTION362 363    # Prefer the first line that looks like an action string364    lines = response_text.splitlines()365    for raw_line in lines:366        line = raw_line.strip()367        if not line:368            continue369        line = ACTION_PREFIX_RE.sub("", line)370        match = ACTION_PATTERN.search(line)371        if match:372            action = match.group(0).strip()373            # Collapse internal whitespace374            action = re.sub(r"\s+", " ", action)375            # If the model tried to click by natural-language description while we376            # only exposed numeric BrowserGym IDs, fallback to the single detected ID.377            return action378 379    # Fall back to searching the whole response380    match = ACTION_PATTERN.search(response_text)381    if match:382        action = match.group(0).strip()383        action = re.sub(r"\s+", " ", action)384        return action385 386    return FALLBACK_ACTION387 388 389def main() -> None:390    client = OpenAI(base_url=API_BASE_URL, api_key=API_KEY)391 392    env = BrowserGymEnv.from_docker_image(393        image="browsergym-env:latest",394        env_vars={395            "BROWSERGYM_BENCHMARK": "miniwob",396            "BROWSERGYM_TASK_NAME": "click-test",397        },398    )399 400    history: List[str] = []401 402    try:403        result = env.reset()404        observation = result.observation405        print(f"Episode goal: {observation.goal}")406 407        for step in range(1, MAX_STEPS + 1):408            if result.done:409                print("Environment signalled done. Stopping early.")410                break411 412            user_prompt = build_user_prompt(step, observation, history)413            user_content = [{"type": "text", "text": user_prompt}]414            screenshot_uri = extract_screenshot_uri(observation)415            if screenshot_uri:416                user_content.append(417                    {418                        "type": "image_url",419                        "image_url": {"url": screenshot_uri},420                    }421                )422 423            messages = [424                {425                    "role": "system",426                    "content": [{"type": "text", "text": SYSTEM_PROMPT}],427                },428                {429                    "role": "user",430                    "content": user_content,431                },432            ]433 434            try:435                completion = client.chat.completions.create(436                    model=MODEL_NAME,437                    messages=messages,438                    temperature=TEMPERATURE,439                    max_tokens=MAX_TOKENS,440                    stream=False,441                )442                response_text = completion.choices[0].message.content or ""443            # pylint: disable=broad-except444            except Exception as exc:  # noqa: BLE001445                failure_msg = f"Model request failed ({exc}). Using fallback action."446                print(failure_msg)447                response_text = FALLBACK_ACTION448 449            action_str = parse_model_action(response_text)450            print(f"Step {step}: model suggested -> {action_str}")451 452            result = env.step(BrowserGymAction(action_str=action_str))453            observation = result.observation454 455            reward = result.reward or 0.0456            error_flag = " ERROR" if observation.last_action_error else ""457            history_line = (458                f"Step {step}: {action_str} -> reward {reward:+.2f}{error_flag}"459            )460            history.append(history_line)461            print(462                "  Reward: "463                f"{reward:+.2f} | Done: {result.done} | Last action error: "464                f"{observation.last_action_error}"465            )466 467            if result.done:468                print("Episode complete.")469                break470 471        else:472            print(f"Reached max steps ({MAX_STEPS}).")473 474    finally:475        env.close()476 477 478if __name__ == "__main__":479    main()You are a senior AI systems engineer specializing in reinforcement learning environments, OpenEnv specification, and agent evaluation systems.480 481Your goal is to design a COMPLETE, HIGH-SCORING OpenEnv environment that can win a competitive evaluation judged on:482- Real-world utility (30%)483- Task & grader quality (25%)484- Environment design (20%)485- Code quality & spec compliance (15%)486- Creativity & novelty (10%)487 488Do NOT repeat the problem statement. Focus on execution.489 490-----------------------------------491PHASE 1: DOMAIN SELECTION (CRITICAL)492-----------------------------------493Propose 3–5 high-impact REAL-WORLD environment ideas (NOT games), such as:494- workflows humans actually perform495- economically or socially valuable tasks496- domains where AI agents are currently weak497 498For each idea:499- Explain why it scores HIGH in real-world utility (not generic reasoning)500- Evaluate feasibility under constraints (2 vCPU, 8GB RAM, <20 min runtime)501- Mention what makes it NOVEL (important for 10% creativity score)502- Identify risks (e.g., grader ambiguity, simulation complexity)503 504Then SELECT the BEST idea with justification.505 506-----------------------------------507PHASE 2: SYSTEM DESIGN (CORE)508-----------------------------------509Design the full OpenEnv environment architecture:510 5111. State Design512- What is the internal state?513- How is it represented (structured objects, DB, memory)?514- How does state evolve across steps?515 5162. Observation Space517- Exact fields (typed, Pydantic-style)518- What the agent sees vs hidden state519- Avoid unnecessary noise520 5213. Action Space522- Define ALL allowed actions523- Must be realistic and constrained (not infinite freedom)524- Format of actions (structured or string-based)525 5264. Step Function Logic527- How actions modify state528- Error handling (invalid actions, edge cases)529- Episode termination conditions530 5315. Reset Logic532- How initial state is generated533- How randomness is controlled for reproducibility534 5356. State() API536- What full state returns vs observation537 538-----------------------------------539PHASE 3: TASK DESIGN (VERY IMPORTANT)540-----------------------------------541Create at least 3 tasks:542- Easy543- Medium544- Hard545 546For EACH task:547- Define clear objective548- Define initial state549- Define constraints550- Explain why difficulty level is appropriate551 552-----------------------------------553PHASE 4: GRADER DESIGN (CRITICAL FOR SCORE)554-----------------------------------555For EACH task:556 557- Define a deterministic grading function → score between 0.0–1.0558- Must NOT be binary only559- Must reward partial progress560 561Include:562- Exact scoring formula563- Edge case handling564- Examples of:565  - perfect score566  - partial score567  - failure568 569Ensure:570- No randomness571- No exploitable loopholes572 573-----------------------------------574PHASE 5: REWARD FUNCTION DESIGN575-----------------------------------576Design step-wise reward shaping:577 578- Immediate rewards for progress579- Penalties for:580  - loops581  - destructive actions582  - inefficiency583 584Explain:585- Why reward helps learning586- How it aligns with final grader587 588-----------------------------------589PHASE 6: BASELINE AGENT DESIGN590-----------------------------------591Design inference.py logic:592 593- How the agent interprets observations594- Prompting strategy595- Action generation constraints596- Handling failures597 598Ensure:599- Deterministic outputs600- Reproducibility601- Works under token + time limits602 603-----------------------------------604PHASE 7: INFRA + DEPLOYMENT605-----------------------------------606Design:607 6081. Folder structure6092. Dockerfile (lightweight, fast startup)6103. openenv.yaml structure6114. Hugging Face Space setup612 613Ensure:614- Passes openenv validate615- Works with docker build/run616- Handles API_BASE_URL, MODEL_NAME, HF_TOKEN617 618-----------------------------------619PHASE 8: WINNING STRATEGY (MOST IMPORTANT)620-----------------------------------621Explain how this solution maximizes score:622 623- Why it will rank high in EACH evaluation category624- What competitors will likely do wrong625- How this design avoids disqualification626 627-----------------------------------628OUTPUT FORMAT629-----------------------------------630Give a clean, structured, implementation-ready blueprint.631 632Avoid fluff. Think like a system designer, not a student.Selected domain: Retail Inventory & Pricing Management. This domain has high real-world impact (major retailers use AI for inventory/pricing633) and maps well to a tractable simulation. It avoids overly-simplistic “toy” tasks and yields rich tradeoffs (profit vs stockouts). It also aligns with agentic trends: the literature notes agents already orchestrate inventory restocks and pricing decisions634. Under 2 vCPU/8 GB, we can model a small store (e.g. 2–5 products, daily time steps) efficiently. This domain is novel in OpenEnv contests (we found no direct analog in existing environments) and covers multi-step planning. Main risk is designing a robust grader and realistic demand model, which we will address in Tasks and Grader phases.635 636Phase 2: System Design6371. State Design638We model a simplified retail store across discrete “day” steps. The internal state includes:639 640Inventory levels: an array/dict of counts for each product.641Cash balance: current store money.642Product data: constant lists of base prices, order costs, holding costs.643Pending orders: shipments en route (if lead times used).644Demand generator state: hidden pattern parameters (e.g. daily mean demand per product).645Day counter: current day/time index.646Representation: A Python class (Pydantic) RetailInventoryState with fields like inventory: Dict[str,int], cash: float, day: int, etc. The state evolves as: on each step, demand is drawn (random or pattern), inventory is decremented by sales, cash is updated from sales and costs, pending orders arrive (increase inventory), and day increments.647 6482. Observation Space649We expose only partial, task-relevant information. The observation includes:650 651inventory: Dict[str,int] – current stock for each product.652cash: float – current cash/budget.653day: int – current time step.654sales_history: Dict[str,int] – last-day sales of each product (or demand forecast).655price: Dict[str,float] – current selling price for each product (if agent can adjust prices).656(Optional) pending_orders: Dict[str,int] – outstanding delivery (only if we want agent to consider lead times).657Hidden in state (not in obs): true demand distribution, hidden exceptions (stockouts, unsatisfied demand). We avoid spurious noise: only relevant features are given. For example, no raw random seed or unnecessary logs. All fields are typed (using Pydantic models or Python dataclasses).658 6593. Action Space660We define a small set of structured actions that mirror real decisions:661 662Order(product: str, quantity: int): Place an order for more units of a product. This is realistic (like restocking). Quantity is bounded (no negative, max reasonable).663SetPrice(product: str, new_price: float): Adjust the selling price of a product. Realistic (sales/promotions).664NoOp(): Do nothing (if agent chooses no change).665Actions are represented as JSON or Pydantic classes (e.g. an Action Union). For example: {"action": "order", "product": "Widget", "quantity": 10}. We constrain values: e.g., cannot order more than some max or when cash insufficient. The action space is discrete with parameters, not infinite text. This ensures realistic constraints and easier parsing. (Alternatively, free-text commands could be used, but structured JSON is more robust under LLM output and spec compliance.)666 6674. Step Function Logic668Each step(action) does:669 670Validate action: If invalid (e.g. ordering negative quantity, unknown product), return error or zero reward with penalty. (Environment should handle gracefully: e.g. ignore invalid action and penalize small negative reward to discourage random outputs.)671Process action:672If Order: subtract order cost (unit_cost*quantity) from cash (or set up pending order if lead time >0).673If SetPrice: update product's selling price (must stay above cost).674If NoOp: do nothing.675Simulate demand: Sample random demand for each product (e.g. Poisson or uniform) based on current pattern. Compute sales = min(demand, inventory).676Update inventory & cash: For each product, inventory -= sales. Add revenue sales * price to cash.677Apply costs/penalties: Deduct holding cost (e.g. small cost per unit left) to penalize overstock. Optionally penalize missed sales (stockout), e.g. reward+0 but track lost opportunity.678Check termination: End episode if day ≥ horizon (e.g. 30 days) or if cash goes below zero. Set done=True.679Increment day: day += 1.680Generate observation and reward: Return new observation (per design) and computed step reward (see Phase 5 for shaping).681No randomness beyond demand sampling. (Random seed fixed on reset ensures reproducibility.) The environment loops until done.682 6835. Reset Logic684reset() initializes a new episode:685 686Set day=0, cash = initial budget (e.g. $1000), initial inventory (random or fixed small stock), default prices (e.g. cost+markup).687Initialize demand pattern (random seed-driven; e.g. demand mean for each product drawn from a range). This hidden pattern is consistent per episode.688Ensure randomness is seeded for reproducibility (np.random.seed(env_seed)).689Return initial observation.690Different tasks will start with different initial states (see Tasks below), but all use a deterministic reset seeded by the task parameters or fixed seed.691 6926. State() API693The state() method returns the full internal state object (all fields, including hidden data) for evaluation or debugging. The observation (via step return) is a subset (inventory, cash, day, etc). This separation ensures the agent only sees observation fields, not hidden pattern or full demand history.694 695Phase 3: Task Design696We propose three graded tasks (Easy, Medium, Hard) that vary in complexity:697 698Easy Task – Single Product Steady Demand.699Objective: Maximize profit for a store selling one product over 7 days.700Initial State: 1 product with cost $5, initial stock 10 units, price $10. Cash = $100. Daily demand fixed at 2 units (known/constant). No lead time for orders.701Constraints: Inventory capacity unlimited. Agent can only order in multiples of 1 and set price once per day.702Difficulty: Very simple scenario. Optimal strategy is to meet demand exactly each day. This tests basic ordering logic and profitability.703 704Medium Task – Two Products, Variable Demand.705Objective: Manage two products (A and B) over 14 days to maximize cumulative profit.706Initial State: Two products with different costs/prices (e.g. A: cost $5, price $10; B: cost $3, price $6). Initial inventory moderate (e.g. 5 of each). Cash = $200. Demand is random: daily demand for A∈[0,4], for B∈[0,3] (unknown but stationary). Lead time 1 day for orders.707Constraints: Inventory capacity per product ≤10 units. Ordering incurs flat shipping fee (e.g. $2 per order placed).708Difficulty: More complex: agent must balance order costs vs stockouts. Pricing might need adjustment for profitability. The stochastic demand requires adaptive policy.709 710Hard Task – Multi-product, Time-varying Demand & Pricing.711Objective: Over 30 days, manage 3 products (C, D, E) with changing demand patterns and dynamic pricing.712Initial State: Three products with distinct costs and initial prices. Initial stock 3 units each, cash = $300. Demand has a trend: e.g. product C demand spikes mid-episode, others decline. Demand is random (Poisson around time-varying means).713Constraints: Inventory capacity small (≤6 per product), and pricing must stay between cost+10% and cost+100%.714Difficulty: High: agent must detect trends and adapt orders/prices. Balancing three products with shifting demand tests planning and partial knowledge.715 716For each task, difficulty scales via number of products, horizon length, and demand uncertainty. Easy tasks allow a straightforward solution, while Hard requires nuanced strategy.717 718Phase 4: Grader Design719For each task we define a deterministic score ∈[0,1]:720 721Grader Metrics: We use normalized profit as the score. Let Profit = final_cash + value_of_inventory - initial_cash (i.e. money earned minus spend). We precompute a high-probability target profit (e.g. through simulation of a strong heuristic or known optimum). Score = min(max(Profit / TargetProfit, 0.0), 1.0). This yields 0 for losses, 1 for meeting/exceeding target, and intermediate for partial success.722 723Task-by-task:724 725Easy Task: Suppose optimal profit = $70. Score = Profit/70.726Perfect: Agent sells all 14 units, Profit=$70 → score 1.0.727Partial: sells 7 units, Profit=$35 → score 0.5.728Failure: misses all demand, Profit<0 → score 0.0 (clamped).729Medium Task: Compute optimal (e.g. $150). Score = Profit/150. Example: profit $120 → 0.8; profit $0 → 0.0.730Hard Task: Target profit e.g. $300. Score = Profit/300. Partial progress yields 0<x<1.731Edge Cases: If Profit < 0, we set score = 0.0. If Profit > Target, we clamp to 1.0 (no reward for overshooting, to avoid exploit). No randomness in grading: the formula is fixed. Even if demand random, scoring is deterministic for a given outcome.732 733No binary grading: We give fractional credit. E.g., selling 50% of potential sales yields ~0.5 score. This rewards partial solutions.734 735This ensures non-binary, continuous evaluation. There are no hidden randomness or ambiguous rules. It directly measures business success (profit) as a single metric.736 737Phase 5: Reward Function Design738We provide dense step-wise rewards aligned with profit:739 740Revenue reward: Each step, reward = (today’s sales revenue) − (order cost) − (holding cost). This aligns immediate reward with profit contributions.741 742Penalties:743 744Stockout Penalty: If demand > inventory, optionally a small penalty (or foregone reward) to discourage stockouts (e.g. lost sale opportunity).745Inventory Holding: We subtract a small cost per unit left (e.g. 0.1 per unit/day) to penalize overstock and encourage lean operations.746Invalid actions: Slight negative reward for invalid/illegal actions.747Efficiency: If episode ends early or overshoots, a final bonus/penalty could be applied (e.g. leftover stock has no value).748Alignment: These rewards push the agent to make profitable decisions each day (sell products and manage costs), which directly correlates with the final profit used in grading. Immediate revenue motivates sales; order costs and holding costs discourage wasteful over-ordering. Over multiple steps, maximizing cumulative reward is equivalent to maximizing profit.749 750Thus, reward shaping provides intermediate feedback that is consistent with the final score (profit). It also penalizes undesirable behaviors (inefficiency, loops, invalid actions), guiding learning even if final success is not immediate.751 752Phase 6: Baseline Agent Design (inference.py)753The baseline inference.py will follow these guidelines:754 755Observation parsing: The agent receives the JSON observation (inventory, cash, day, prices). It summarizes recent sales perhaps from sales_history. The system prompt will describe the task (e.g. “You manage store inventory to maximize profit…”).756 757Prompting Strategy:758 759Use a deterministic system prompt (templates with environment description, valid actions).760Include a few example turns if needed (like ordering when low stock).761Emphasize output format: “Respond with exactly one action in JSON {action:...,product:...,quantity:...} or {action: 'set_price',...}.”762Possibly list valid products and their names in the prompt.763Action generation: The agent uses the OpenAI client with api_base_url, model_name, etc., as mandated. It calls client.completions.create with temperature=0.0 for determinism and low max_tokens (e.g. 50). The stop sequence ensures only the action JSON is output (we can trim any extra text).764 765Constraints: The prompt will include current cash and inventory so the model “knows” not to overspend. We may instruct it “Don’t order more than cash allows” or we check after generation.766 767Failure handling: If the LLM outputs invalid JSON or illegal action, the agent will default to NoOp() (the fallback). This ensures robustness.768 769Determinism & Limits: Temperature 0, no randomness. Care to fit under time/token budget (observations are small). The code uses OPENAI client as specified. No external calls beyond OpenAI. It uses API_BASE_URL, MODEL_NAME, HF_TOKEN from env variables.770 771Phase 7: Infrastructure & Deployment772Folder Structure:773 774graphql775Copy776retail_inventory_env/777├── README.md778├── openenv.yaml779├── pyproject.toml         # env dependencies780├── environment/781│   ├── __init__.py782│   ├── retail_env.py      # implements Environment subclass783│   └── models.py          # Pydantic Action/Observation/State models784├── server/785│   ├── app.py             # FastAPI app loading the environment786│   └── Dockerfile         # container build787└── inference.py           # baseline agent script788Dockerfile: Lightweight (e.g. Python 3.10-slim). Example steps:789 790sql791Copy792FROM python:3.10-slim793WORKDIR /app794COPY pyproject.toml .795RUN pip install --no-cache-dir --upgrade pip \796    && pip install --no-cache-dir -e .797# If using uvicorn for OpenEnv server:798CMD ["uvicorn", "server.app:app", "--host", "0.0.0.0", "--port", "8000"]799This builds fast and installs only necessary deps. We ensure the container is minimal (no heavy libs besides FastAPI).800 801openenv.yaml: Manifest specifying environment metadata:802 803yaml804Copy805name: "RetailInventoryEnv"806description: "Simulates store inventory management and pricing tasks."807version: "0.1.0"808author: "YourTeamName"809variables:810  - API_BASE_URL811  - MODEL_NAME812  - HF_TOKEN813requirements:814  - python >=3.10815  - fastapi816  - pydantic817Key fields: name, version, author, required env variables (as mandated). The openenv validate tool will check this.818 819Hugging Face Space Setup: We create a Space with environment code (pointing to this repo). The openenv push command packages this structure. The Space Dockerfile (via uvicorn) is set to serve the environment endpoint. We ensure the Space README gives instructions to connect via RetailInventoryEnv. The inference script should be runnable locally using provided API_BASE_URL.820 821All components are spec-compliant: pyproject.toml lists dependencies, openenv.yaml is valid, and server/app.py creates the FastAPI app (following templates). On docker build && docker run, the environment server starts without errors. We will test with openenv validate and sample client.reset() in CI.822 823Phase 8: Winning Strategy824This design maximizes the scoring criteria:825 826Real-world utility (30%): We chose a high-impact domain. Retail inventory and pricing are critical in commerce827. Solving this helps businesses reduce waste and increase profit. The scenario is more compelling than toy puzzles. Citing real industry usage828 shows practicality.829 830Task & grader quality (25%): We provide three graded tasks of increasing complexity, with clear objectives and constraints. The grader is fully deterministic and continuous (profit-based), rewarding partial progress. Unlike binary or hackable metrics, our profit formula is transparent and robust to tie-breaking. Example scoring demonstrates non-trivial partial credit, satisfying the requirement for non-binary grading.831 832Environment design (20%): We offer a complete, well-structured environment architecture: formal state, observation, and action specs with Pydantic models. The step/reset logic and API design are clearly defined. This thoroughness avoids ambiguities and meets OpenEnv best practices. For instance, the separation of observed vs hidden state ensures the challenge remains non-trivial.833 834Code quality & spec compliance (15%): The plan includes a proper folder layout, pyproject.toml, a lean Dockerfile, and an openenv.yaml manifest. These choices ensure passing openenv validate and quick container startup. Using lightweight base images and minimal dependencies keeps the environment within resource limits. The inference script will follow the official OpenAI client usage (using API_BASE_URL, HF_TOKEN, etc.) with fixed seeds for reproducibility.835 836Creativity & novelty (10%): This environment is creative: it models a dynamic store that an agent must run, which is more novel than many hackathon entries (e.g. not just calendar or coding). Combining inventory decisions with pricing and multi-day demand is inventive. We also included features like lead time and holding costs, adding realism. Compared to a “to-do list” or text puzzle, this truly simulates a small business.837 838Why competitors might fall short: Many may choose simpler or game-like domains (email sorting, to-dos, trivia), which earn low “real-world utility” points. Others might implement binary success tasks (score 0/1), which fail the grader criterion. By contrast, our design explicitly rewards nuance (partial profit) and scale (multiple products and days). Competitors might also neglect spec compliance (broken openenv.yaml or slow Docker), whereas we emphasize clean CI/CD integration.839 840Avoiding disqualification: We adhere to all requirements:841 842Use of environment variables (API_BASE_URL, MODEL_NAME, HF_TOKEN).843All model calls use the OpenAI client (not raw requests).844The inference script is deterministic (temperature=0).845The environment is reproducible (seeded randomness).846Everything runs within resource/time limits (no heavy ML model, just simple math loops).847openenv.yaml and Docker setup ensure smooth deployment.848In summary, this blueprint delivers a complete, polished environment with high real-world relevance, robust tasks and grading, and solid engineering practices. It addresses each rubric category directly and avoids common pitfalls, positioning it as a top-scoring solution.849