CoolFace
Apppublic

RohanExploit/Meta-hackathon

sourceHugging Faceupdated 5mo agoView on Hugging Face
0likes
App README

πŸͺ Multi-Channel Retail Environment with Disruption Recovery

OpenEnv Scaler Hackathon Submission β€” A production-grade, OpenEnv-compliant environment that challenges AI agents to manage a realistic multi-channel retail operation under supply chain disruptions, demand shocks, and stochastic supplier behavior.

![Live on HF Spaces](https://huggingface.co/spaces/RohanExploit/Meta-hackathon) ![Python 3.10+](https://www.python.org/) ![OpenEnv Compliant]() ![Tests]()


🎯 What Problem Does This Solve?

Retail operations managers face a daily challenge that no toy environment captures: simultaneously optimizing pricing, inventory, and promotions across multiple customer segments while dealing with unpredictable supply chain disruptions.

This environment faithfully simulates the real-world tradeoffs:

  • β€”A demand collapse hits your bestseller β€” do you slash prices or run a promotion?
  • β€”Your supplier delivers only 80% of orders β€” how much buffer stock do you carry?
  • β€”Luxury customers will pay 2Γ— but buy less β€” how do you allocate limited inventory between segments?
  • β€”Holding costs eat into margins β€” when exactly should you reorder?

These are genuine decisions that cost retailers billions annually. This is not a game.


πŸ—οΈ Architecture

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                     FastAPI Server (server/app.py)           β”‚
β”‚  POST /reset Β· POST /step Β· GET /state Β· POST /evaluate     β”‚
β”‚  GET /tasks Β· GET /health Β· POST /live/start Β· /stop Β· ...   β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚                     Core Environment                         β”‚
β”‚  environment/retail_env.py   ← MultiChannelRetailEnv         β”‚
β”‚  environment/models.py       ← Pydantic: Action, Obs, State β”‚
β”‚  environment/grader.py       ← Deterministic scoring [0,1]  β”‚
β”‚  environment/tasks.py        ← 5 difficulty tiers            β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚              Baseline Agent (inference.py)                    β”‚
β”‚  OpenAI-compatible client β†’ observe β†’ reason β†’ act loop      β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

πŸ•ΉοΈ Action Space

The agent selects one action per step from five typed Pydantic models:

ActionJSON SchemaPurpose
Allocate{"action": "allocate", "product": "str", "luxury_units": int, "budget_units": int}Assign inventory between luxury (high-margin) and budget (high-volume) segments
Set Price`{"action": "set_price", "product": "str", "segment": "luxury\budget", "new_price": float}`Dynamic pricing within bounds β€” higher price reduces demand but increases margin
Order{"action": "order", "product": "str", "quantity": int}Purchase from supplier β€” cost deducted immediately, delivery has stochastic lead time
Promote{"action": "promote", "product": "str", "budget_allocated": float}Run a promotional campaign β€” upfront cash cost to stimulate demand
NoOp{"action": "noop"}Hold current strategy β€” wait and observe

πŸ‘οΈ Observation Space

The agent receives a partial observation (true demand patterns are hidden):

python
RetailObservation(
    day: int,                    # Current day [0, horizon)
    cash: float,                 # Available funds
    inventory: Dict[str, int],   # Current stock by product
    recent_demand_luxury: Dict,  # Rolling 3-day average of realized luxury demand
    recent_demand_budget: Dict,  # Rolling 3-day average of realized budget demand
    recent_stockouts: Dict,      # Per-product stockout count (steps with unmet demand)
    prices_luxury: Dict,         # Current luxury prices
    prices_budget: Dict,         # Current budget prices
    disruption_active: bool,     # Is a disruption currently happening?
    disruption_severity: float,  # Severity [0.0–1.0]
    market_confidence: float,    # Decaying confidence indicator [0.0–1.0]
)

The agent must infer hidden demand patterns from noisy signals β€” this is a partially observable environment by design.


πŸ“Š Tasks (5 Difficulty Tiers)

TaskHorizonProductsSupplier ReliabilityKey ChallengeBaseline Profit
easy10 days1100%Stable demand, no disruptions$50
medium_simple14 days295%Variable demand, occasional disruptions$100
medium_challenge14 days288%Supply delays, demand elasticity shifts$80
hard21 days385%Frequent disruptions, tight margins$120
expert30 days480%Extreme chaos: multi-product, high variance$200

Each task has a fixed seed for reproducible evaluation.


πŸ† Grading System

The grader (environment/grader.py) is deterministic, transparent, and stateless. It computes a composite score in [0.0, 1.0]:

score = 0.5 Γ— profit_score + 0.3 Γ— fill_rate_score + 0.2 Γ— efficiency_score
ComponentFormulaWeightRationale
Profit Scoreclamp(profit / baseline_profit, 0, 1)50%Primary business metric
Fill Rate Scoreclamp(total_sales / total_demand, 0, 1)30%Customer satisfaction proxy
Efficiency Scoreclamp(1 - holding_cost / max_possible_holding_cost, 0, 1)20%Capital efficiency

Anti-Gaming Guardrail

If fill_rate < 0.6, the final score is halved. This prevents agents from exploiting high-margin strategies that ignore customer demand.

Multi-Seed Aggregation

The /evaluate endpoint supports multi-seed evaluation with optional variance penalty:

adjusted_score = mean_score - variance_penalty Γ— variance(seed_scores)

πŸš€ Installation & Quick Start

Prerequisites

  • β€”Python 3.10+ (if running locally)
  • β€”Docker (optional, if running via containerized setup)
  • β€”Hugging Face Token / API Key (to run the baseline inference agent)

Local Installation & Development

bash
# Clone the repository
git clone https://github.com/RohanExploit/Meta-hackathon.git
cd Meta-hackathon

# Setup
python -m venv .venv
source .venv/bin/activate          # Linux/Mac
# .venv\Scripts\activate           # Windows

pip install --upgrade pip
pip install -e .

# Start the environment server
uvicorn server.app:app --host 0.0.0.0 --port 8000

Run Baseline Inference

In a second terminal:

bash
export API_BASE_URL=https://router.huggingface.co/v1
export MODEL_NAME=NousResearch/Nous-Hermes-2-Mistral-7B-DPO
export OPENAI_API_KEY=<your-hf-token>

python inference.py

Docker

bash
docker build -t retail-env .

docker run -p 8000:8000 \
  -e API_BASE_URL="https://router.huggingface.co/v1" \
  -e MODEL_NAME="NousResearch/Nous-Hermes-2-Mistral-7B-DPO" \
  -e OPENAI_API_KEY="<token>" \
  retail-env

Hugging Face Spaces

This repository is deployed live at [huggingface.co/spaces/RohanExploit/Meta-hackathon](https://huggingface.co/spaces/RohanExploit/Meta-hackathon). The Space auto-builds from the root Dockerfile and starts serving the API immediately.


πŸ“‘ API Reference

All endpoints are served by FastAPI with automatic OpenAPI docs at /docs.

EndpointMethodDescription
/healthGETHealth check β†’ {"status": "healthy"}
/tasksGETList all 5 tasks with metadata
/resetPOSTReset environment to a task: {"task_name": "easy", "seed": 42}
/stepPOSTExecute action: {"action": {"action": "order", "product": "Product_A", "quantity": 5}}
/stateGETFull internal state + episode metrics (for debugging and scoring)
/evaluatePOSTScore an episode summary or batch of summaries
/live/startPOSTStart real-time continuous stepping (heuristic or noop mode)
/live/stopPOSTStop real-time stepping
/live/statusGETReal-time runner status
/live/latestGETLatest real-time tick observation
/live/streamGETServer-Sent Events stream of live runner ticks (real-time push, no polling needed)
/dashboardGETModern visual dashboard UI (Chart.js charts, manual actions, live runner)
/GETClassic terminal-style UI

Example: Full Episode

bash
# 1. Reset to easy task
curl -s -X POST http://localhost:8000/reset \
  -H "Content-Type: application/json" \
  -d '{"task_name": "easy", "seed": 42}'

# 2. Take an action
curl -s -X POST http://localhost:8000/step \
  -H "Content-Type: application/json" \
  -d '{"action": {"action": "order", "product": "Product_A", "quantity": 3}}'

# 3. Evaluate
curl -s -X POST http://localhost:8000/evaluate \
  -H "Content-Type: application/json" \
  -d '{"summary": {"profit": 80, "baseline_profit": 100, "total_demand": 100, "total_sales": 75}}'

πŸ§ͺ Testing & Validation

Compliance Tests (6 tests)

bash
python -m pytest tests/test_compliance.py -v

Validates: OpenEnv YAML contract, HF Spaces config, task count, API endpoints, grader bounds, anti-gaming behavior.

Smoke Tests

bash
python test_env.py    # Environment lifecycle
python test_api.py    # API simulation

Health Check (all-in-one)

bash
python scripts/health_check.py

Runs compile checks β†’ test suite β†’ smoke scripts β†’ pre-submission checks.

Pre-Submission Check

bash
python scripts/pre_submission_check.py

Validates: task count β‰₯ 3, inference env vars, Docker availability, OpenEnv CLI.

Reproducibility Benchmark

bash
# Requires running server + API credentials
python scripts/benchmark_inference.py

Runs inference twice, compares scores for determinism.

Verified Baseline Scores

Tested with llama-3.1-8b-instant via Groq (seed=42):

TaskScoreProfit ScoreFill RateEfficiencyFinal Cash
easy0.430.001.000.67$57.60
medium_simple0.951.001.000.77$437.18
medium_challenge0.911.000.740.92$609.46
hard0.421.000.470.95$917.84
expert0.391.000.260.97$987.71
Mean0.62
The agent achieves 100% fill rate on easy/medium tasks. Harder tasks have lower fill rates due to multi-product inventory splitting across 3-4 products with a single action per step.

πŸ“‚ Repository Structure

Meta-hackathon/
β”œβ”€β”€ Dockerfile                  # Root-level Docker config (HF Spaces uses this)
β”œβ”€β”€ README.md                   # This file
β”œβ”€β”€ app.spaces.yaml             # HF Spaces auto-config (sdk: docker, tag: openenv)
β”œβ”€β”€ openenv.yaml                # OpenEnv environment metadata
β”œβ”€β”€ pyproject.toml              # Dependencies + entry points
β”œβ”€β”€ uv.lock                     # Locked dependency versions
β”œβ”€β”€ inference.py                # Baseline LLM agent (OpenAI-compatible)
β”‚
β”œβ”€β”€ environment/                # Core environment package
β”‚   β”œβ”€β”€ __init__.py
β”‚   β”œβ”€β”€ models.py               # Pydantic: RetailAction, RetailObservation, RetailState
β”‚   β”œβ”€β”€ retail_env.py           # MultiChannelRetailEnv (reset, step, state)
β”‚   β”œβ”€β”€ grader.py               # Deterministic scoring [0, 1]
β”‚   └── tasks.py                # 5 task definitions (easy β†’ expert)
β”‚
β”œβ”€β”€ pipeline/                   # NVIDIA-inspired RAG pipeline (optional layer)
β”‚   β”œβ”€β”€ config.py               # Centralized model registry + tuning knobs
β”‚   β”œβ”€β”€ safety.py               # Llama Guard 3 input/output guardrails
β”‚   β”œβ”€β”€ router.py               # NIM-style Router Agent (query classification)
β”‚   β”œβ”€β”€ retriever.py            # Two-stage: FAISS dense retrieval + LLM reranking
β”‚   β”œβ”€β”€ chain.py                # LCEL orchestrator (safetyβ†’routerβ†’retrievalβ†’gen)
β”‚   └── ingest.py               # SemanticChunker document ingestion
β”‚
β”œβ”€β”€ server/                     # FastAPI server package
β”‚   β”œβ”€β”€ __init__.py
β”‚   β”œβ”€β”€ app.py                  # 12 API endpoints + LiveRunner + SSE stream
β”‚   └── ui/
β”‚       β”œβ”€β”€ index.html          # Terminal-style web UI (/)
β”‚       β”œβ”€β”€ dashboard.html      # Modern visual dashboard (/dashboard) β€” Chart.js charts, live runner
β”‚       └── chart.umd.min.js    # Bundled Chart.js (served at /static/chart.js)
β”‚
β”œβ”€β”€ tests/
β”‚   └── test_compliance.py      # 6 compliance tests
β”‚
β”œβ”€β”€ scripts/
β”‚   β”œβ”€β”€ health_check.py         # All-in-one validation
β”‚   β”œβ”€β”€ pre_submission_check.py # Rubric-mapped checks
β”‚   β”œβ”€β”€ benchmark_inference.py  # Reproducibility benchmark
β”‚   └── run_tests.py            # Isolated test runner
β”‚
β”œβ”€β”€ test_env.py                 # Environment smoke test
β”œβ”€β”€ test_api.py                 # API smoke test
└── .github/workflows/ci.yml   # GitHub Actions CI

πŸ”‘ Key Design Decisions

Why Multi-Segment Pricing?

Real retailers optimize per customer tier. Luxury customers accept premium prices with lower volume; budget customers demand high volume at thin margins. This creates a genuine tradeoff that doesn't exist in single-price environments.

Why Disruptions?

Supply chain disruptions (demand collapses, supply delays, demand spikes) occur with ~15% probability per step. They test whether agents can adapt and recover β€” a capability critical in real-world retail operations.

Why Transparent Grading?

The grader formula is fully documented and deterministic. Agents can learn that profit matters most (50% weight), customer satisfaction is essential (30%), and capital efficiency rounds out performance (20%). No hidden weights.

Why Partial Observability?

Agents see demand signals, not true demand distributions. This mirrors reality: retailers observe sales data and stockout counts, not the underlying customer arrival process.


πŸ“ˆ Expected Baseline Scores

TaskHeuristic AgentLLM Agent (7B)Optimal Estimate
easy~0.85~0.90~0.95
medium_simple~0.55~0.70~0.85
medium_challenge~0.45~0.60~0.80
hard~0.30~0.45~0.70
expert~0.20~0.35~0.60

πŸ”„ Reproducibility

  • β€”All randomness is seeded (np.random.seed, random.seed) via task config
  • β€”Inference uses temperature=0 for deterministic LLM output
  • β€”Each task has a fixed grading seed for reproducible evaluation
  • β€”Episode metrics are fully logged in the terminal step info
  • β€”Docker build is deterministic with pinned dependencies (uv.lock)

βš™οΈ Environment Variables

VariableRequiredDescription
API_BASE_URLFor inferenceLLM API endpoint (e.g., https://router.huggingface.co/v1)
MODEL_NAMEFor inferenceModel identifier (e.g., NousResearch/Nous-Hermes-2-Mistral-7B-DPO)
OPENAI_API_KEYFor inferenceAPI key / HF token for authentication
PORTOptionalServer port (default: 8000)
ENV_BASE_URLFor inferenceEnvironment server base URL (default: http://127.0.0.1:8000)

πŸ“‹ Hackathon Checklist

RequirementStatusEvidence
Real-world task simulationβœ…Multi-channel retail with disruptions
Full OpenEnv spec (typed models, step/reset/state, openenv.yaml)βœ…Pydantic models + 10 API endpoints
Minimum 3 tasks (easy β†’ hard, scores 0.0–1.0)βœ…5 tasks with graded difficulty
Meaningful reward function with partial progress signalsβœ…Per-step: revenue βˆ’ stockout penalty βˆ’ holding costs
Baseline inference script with reproducible scoresβœ…inference.py with seeded episodes
Deploy to HF Spaces + working Dockerfileβœ…Live Space
README with environment description, spaces, setupβœ…This document
Agent graders (scores 0.0–1.0)βœ…environment/grader.py β€” deterministic, bounded
CI/CDβœ….github/workflows/ci.yml

πŸ“œ License

MIT


Built for the Meta OpenEnv Scaler Hackathon 2026.