Amulya001/rl-testing-env

RL-Testing-Env

An OpenEnv environment for training agents to write effective software tests

  

Overview

RL-Testing-Env fills a gap in the OpenEnv ecosystem: there are no environments that train agents on code quality tasks. Writing good tests is one of the highest-leverage skills a software AI agent can have—yet no existing benchmark measures it in a grounded, executable way.

This environment presents agents with real Python code containing injected bugs and rewards them for writing pytest suites that catch those bugs, achieve coverage targets, and correctly triage regressions. Every reward signal is grounded in actual test execution, not heuristics.

Unlike synthetic benchmarks, agents must handle realistic challenges: subtle math errors, off-by-one mistakes, incorrect conditionals, and distinguishing real regressions from intentional behavior changes.

Environment Description

RL-Testing-Env simulates the real-world workflow of a software quality engineer. In production settings, QA engineers must:

1. 1.Write unit tests that catch bugs before they reach production
2. 2.Achieve coverage targets while identifying which components are most buggy
3. 3.Triage test failures after code changes—distinguishing real regressions from intentional behavior changes

The environment provides three progressively difficult tasks that mirror these responsibilities. Agents receive Python source code with injected bugs and must submit pytest test files. A subprocess-based grader executes the tests securely, measures coverage, and computes rewards based on bugs caught and correct classifications.

This environment is particularly valuable for:

• —Evaluating LLM code generation capabilities in a grounded, measurable way
• —Training agents that can assist developers with test-driven development
• —Benchmarking progress on software engineering automation

Observation Space

Each observation provides the agent with everything needed to write effective tests:

Action Space

Agents interact with the environment through structured actions:

Action Types

• —`submit_tests`: Execute the provided pytest code against the buggy code and receive a reward
• —`view_coverage`: Check current coverage without consuming a grading attempt
• —`done`: End the episode early (useful when agent is satisfied with results)

Tasks

Task 1: Unit Test Writer

Objective: Write pytest tests for a calculate_order_total(items, discount, tax_rate) function that computes order totals with discounts and tax.

Challenge: The function contains exactly 3 bugs selected from a pool of 6 possible bugs:

• —Wrong arithmetic operators (+ instead of *)
• —Incorrect divisors (/10 instead of /100)
• —Inverted operations (add instead of subtract)
• —Tax calculated on wrong amount
• —Rounding precision errors

Grader Logic:

1. 1.Compile test code (syntax check)
2. 2.For each of the 3 bugs, run tests against a version with only that bug
3. 3.A bug is "caught" if at least one test fails because of it

Expected Scores: 0.4–1.0 for competent agents

Task 2: Coverage Audit

Objective: Test an Inventory class with 5 methods (add_item, remove_item, calculate_total, apply_discount, get_summary) while achieving 80%+ line coverage AND identifying which functions contain bugs.

Challenge: 2–3 of the 5 functions have bugs. The agent must:

• —Write tests that cover most code paths
• —Ensure tests fail on buggy functions (identifying them)
• —Avoid false positives (tests failing on correct functions)

Grader Logic:

1. 1.Run pytest --cov to measure line coverage
2. 2.Parse which test functions fail to identify "flagged" functions
3. 3.Compare flagged functions against actual buggy functions

Expected Scores: 0.3–0.85 for competent agents

Task 3: Regression Audit

Objective: Analyze changes between v1 and v2 of an authentication module. Classify each failing test as either a regression (bug introduced) or intentional change (test needs updating).

Challenge:

• —v1 code worked correctly with 8 passing tests
• —v2 contains 2 real regressions and 1 intentional behavior change
• —Agent must classify failures, write new tests for regressions, and update tests for intentional changes

Submission Format:

# FAILING_TESTS: test_name1, test_name2
# CLASSIFICATION: test_name1=regression
# CLASSIFICATION: test_name2=intentional_change

def test_regression_new():
    """New test catching the regression."""
    ...

def test_name2_updated():
    """Updated test for intentional change."""
    ...

Grader Logic:

1. 1.Run original v1 tests against v2 to confirm which fail
2. 2.Parse agent's classification comments
3. 3.Run agent's new/updated tests against v2

Expected Scores: 0.2–0.75 for competent agents

Reward Function

AutoTest-Env uses partial credit design to provide dense learning signals:

Unit Test Writer (Easy)

Maximum: 1.0 (all 3 bugs caught)

Coverage Audit (Medium)

Maximum: 1.0 (80%+ coverage, all bugs identified, no false positives)

Regression Audit (Hard)

Maximum: 1.0 (all failures identified and correctly classified)

Setup Instructions

Prerequisites

• —Python 3.11+
• —Docker (optional, for containerized deployment)

Installation

# Clone the repository
git clone https://github.com/AmulyaSKumar/rl-testing-env
cd autotest-env

# Install dependencies
pip install openenv-core
pip install -e .

Running with Docker (Recommended)

# Build the container
docker build -t autotest-env .

# Run the environment server
docker run -p 7860:7860 autotest-env

Running Locally

# Start the environment server
uvicorn server.app:app --host 0.0.0.0 --port 7860

Running Inference

# Option 1: Using HuggingFace (required for OpenEnv evaluation)
export HF_TOKEN="your-huggingface-token"
export API_BASE_URL="https://router.huggingface.co/v1"
export MODEL_NAME="Qwen/Qwen2.5-72B-Instruct"
python inference.py --seed 42

# Option 2: Using OpenAI API (for local testing)
export OPENAI_API_KEY="your-openai-key"
python inference.py --seed 42

# Run specific task only
python inference.py --seed 42 --task unit_test_writer

# Run all tasks
python inference.py --seed 42

Baseline Scores

Reproducible Baseline (seed=42)

To reproduce exactly: ``bash HF_TOKEN=your_token python inference.py --seed 42 ``

Multi-Seed Baseline (averaged across seeds 42, 123, 456)

Expected Scores by Model Class

Tasks are deterministic with the same seed. Multi-seed evaluation recommended for robust comparison.

Environment Variables

*HFTOKEN is required for OpenEnv evaluation. OPENAIAPI_KEY can be used as fallback for local testing.

API Reference

Endpoints

Python Client

A Python client SDK is available in client.py for programmatic interaction without running inference.py:

from client import AutoTestEnvClient
from models import TestAction

client = AutoTestEnvClient("http://localhost:7860")
result = client.reset(task_id="unit_test_writer", seed=42)

Example Usage

import requests

# Start a new episode
response = requests.post("http://localhost:7860/reset", json={
    "task_id": "unit_test_writer",
    "seed": 42
})
observation = response.json()["observation"]

# Submit tests
response = requests.post("http://localhost:7860/step", json={
    "action_type": "submit_tests",
    "test_code": """
import pytest
from order_module import calculate_order_total

def test_basic():
    result = calculate_order_total([(10, 2)], 0, 0.0)
    assert result == 20.0
""",
    "notes": "Testing basic multiplication"
})
result = response.json()
print(f"Reward: {result['reward']}, Done: {result['done']}")

Output Format

The inference script produces standardized output:

[START] task=unit_test_writer env=autotest-env model=Qwen/Qwen2.5-72B-Instruct
[STEP] step=1 action=import pytest... reward=0.40 done=false error=null
[STEP] step=2 action=def test_edge()... reward=0.30 done=false error=null
[END] success=true steps=2 score=0.700 rewards=0.40,0.30

[START] task=coverage_audit env=autotest-env model=Qwen/Qwen2.5-72B-Instruct
...

Security

• —All test execution happens in subprocesses with timeouts (10–20 seconds)
• —No use of exec() or eval() on agent-submitted code
• —Docker container runs as non-root user
• —Temporary files are automatically cleaned up

Grader Design & Exploit Resistance

Each grader is designed to be exploit-proof:

All tasks clamp rewards to [0.0, 1.0] and penalize false positives to discourage gaming strategies.

License

MIT License. See LICENSE for details.