Debdeep123/capability_forge_codeops

CapabilityForge: CodeOps

CapabilityForge: CodeOps is an OpenEnv-compatible RL environment that acts as an adaptive curriculum generator for software engineering tasks. It focuses on the real-world task of Code Review and Bug Fixing.

Rather than presenting static, one-shot code generation tasks, CodeOps simulates the iterative debugging process. The environment provides multi-step reasoning tasks and adjusts difficulty based on a moving average of the model's success rate, ensuring the model trains in its "Zone of Proximal Development."

Why this matters (Real-world utility)

Frontier models are increasingly used for complex debugging and code synthesis. However, static benchmarks saturate quickly and don't reflect the iterative nature of software development. CodeOps bridges this gap by providing an interactive environment where an agent proposes a fix, the environment compiles and tests it using an execution grader, and the agent uses the resulting error traces to try again—exactly like a human engineer.

Action & Observation Spaces

Observation

• —task_prompt: The instructions for the current debugging task.
• —buggy_code: The Python code containing the bug.
• —execution_feedback: Feedback or error trace from running the previous attempt's code (None on first step).
• —hint: A strategic hint provided after multiple failed attempts to guide the agent.

Action

• —reasoning: Step-by-step reasoning trace explaining the bug and the proposed fix.
• —corrected_code: The complete, corrected raw Python code.

Tasks and Graders

The environment contains 3 distinct tasks (which scale dynamically based on the agent's win rate):

1. 1.Easy (Syntax/Type Error): Fix an arithmetic logic error.
2. 2.Medium (Logic Error): Fix a flawed loop condition.
3. 3.Hard (Algorithm/Edge Case): Fix a sliding window string algorithm.

Grader: The programmatic execution grader (exec + hidden assertions) tests the code safely.

• —0.0: Code fails to execute.
• —0.5: Code executes but fails assertions (Partial Progress).
• —1.0: Code passes all assertions.

Rewards are shaped to include execution score + process rewards (solving in fewer attempts, correct formatting).

Setup Instructions

1. 1.Clone the repository and cd into it.
2. 2.Install dependencies:

   pip install "openenv-core[cli]" fastapi uvicorn pydantic openai

1. 1.Run the environment locally:

   uvicorn server.capability_forge_codeops_environment:app --reload

Baseline Inference

The baseline script uses the OpenAI client to run the agent against the environment.

export OPENAI_API_KEY=your_api_key
python inference.py

Docker Deployment

To build and run the Dockerized HF Space:

docker build -t capability_forge_codeops .
docker run -p 8000:8000 capability_forge_codeops