MeenalSinha/cloud-finops-optimizer

Cloud FinOps Optimizer

An OpenEnv-compliant reinforcement learning environment where an AI agent acts as a Cloud FinOps Engineer. The agent must reduce cloud infrastructure costs safely by terminating waste, right-sizing over-provisioned compute, and applying reserved pricing — without disrupting mission-critical systems.

Problem Motivation

Cloud cost management is a critical discipline. Organizations routinely overspend 20-40% on cloud infrastructure through idle resources, over-provisioned instances, and missed pricing opportunities such as reserved instances.

Human FinOps engineers must reason under constraints, respect operational boundaries, and make sequential decisions with downstream consequences. This environment models that decision-making process faithfully, making it a high-value benchmark for training and evaluating AI agents on real-world planning tasks.

Project Structure

cloud-finops-env/
├── models.py              Typed Pydantic models (FinOpsAction, FinOpsObservation, FinOpsState)
├── client.py              EnvClient subclass (WebSocket + HTTP)
├── inference.py           Baseline inference script (uses OpenAI client)
├── openenv.yaml           OpenEnv metadata manifest
├── pyproject.toml         Package metadata (pip-installable from HF Space)
├── requirements.txt       Python dependencies
├── validate.py            Pre-submission validator
├── README.md              This file
└── server/
    ├── app.py             FastAPI server (create_fastapi_app)
    ├── environment.py     Core environment logic and graders
    └── Dockerfile         Container definition

Action Space

The agent selects one action per step. All actions serialize to a JSON object.

Note on Reasoning: Every action accepts an optional reasoning field. High-quality justifications significantly improve the episode grade.

Observation Space

Each step returns a FinOpsObservation with the following fields:

Each CloudResource has advanced metadata:

Reward Function

Tasks

Task 1 — Waste Cleanup (Easy)

Identify and terminate the 4 idle resources hidden among active ones. Focus on cpu_utilization == 0.0.

Task 2 — Performance-Aware Rightsizing (Medium)

Downsize 5 over-provisioned EC2 instances. Challenge: Resizing too aggressively will breach sla_max_cpu constraints.

Task 3 — Fleet-Wide Tactical Strategy (Hard)

Execute a multi-stage plan: terminate waste, reserve high-load servers, and right-size others while managing Temporal Cooldowns (actions take time to stabilize).

Grader Quality & Explainability

Each task includes an Episode Grader that returns a score in the strict (0.01, 0.99) range.

The final score is composed of:

1. 1.Task Success (90%): Absolute cost reduction and SLA compliance.
2. 2.Explainability Bonus (10%): The agent's reasoning is evaluated for keyword depth, logical structure ("Because X, I did Y"), and contextual awareness of resource IDs.

API Endpoints

Setup and Usage

Install from HF Space

pip install git+https://huggingface.co/spaces/username/cloud-finops-optimizer

Connect to a running Space

from client import FinOpsEnv, FinOpsAction

with FinOpsEnv(base_url="https://username-cloud-finops-optimizer.hf.space").sync() as env:
    result = env.reset(task_id="task1")
    result = env.step(FinOpsAction(action_type="terminate", resource_id="ebs-001"))
    print(result.reward, result.done)

Run locally

pip install -r requirements.txt

# Start the server
uvicorn server.app:app --host 0.0.0.0 --port 7860 --reload

# Check health
curl http://localhost:7860/health

# Run baseline inference
export OPENAI_API_KEY=sk-...
python inference.py

Docker

docker build -t cloud-finops-optimizer -f server/Dockerfile .
docker run -d -p 7860:7860 cloud-finops-optimizer
curl http://localhost:7860/health

Deploy to HF Spaces

openenv push --repo-id username/cloud-finops-optimizer

Validate before submitting

python validate.py

Environment Variables

Baseline Scores

Scores produced by gpt-4o-mini at temperature=0:

Average: 0.5333

Scores are reproducible at temperature=0.

Design Notes

Determinism. All resource lists are hard-coded constants, not randomly generated. The same agent behavior always produces the same grade.

Dense rewards. Every action returns a non-zero signal. Cost saved per action, idle bonuses, strategy bonuses, and penalties are applied immediately. This prevents sparse-reward problems during RL training.

Mission-critical safety. Terminating a critical resource applies a -1.0 reward and is penalized in the grade. This models the real-world constraint that FinOps actions must not disrupt production.

Resource dependencies. Some resources declare dependency_ids. Breaking an active dependency applies an additional -0.20 penalty per affected resource, encouraging the agent to reason about downstream effects.

Hardware. The environment runs on 2 vCPU / 8 GB RAM. No ML model is loaded. Inference completes well within the 20-minute runtime limit.