svmtejas84/OpenEnv-SOC2-Auditor

audit-gym-v1

A professional OpenEnv environment designed for SOC2 compliance audit training.

This environment simulates realistic audit workflows where agents inspect policy and control evidence, identify risky access patterns, and submit findings with dense partial-credit rewards.

Quick Start

1. 1.Create and activate a virtual environment:

python -m venv .venv
source .venv/bin/activate

1. 1.Install dependencies:

pip install -e .

1. 1.Start the API server:

uvicorn main:app --host 0.0.0.0 --port 7860

1. 1.Validate the environment with OpenEnv:

openenv validate

1. 1.Run the inference script (optional):

python inference.py

Inference Environment

inference.py reads two separate endpoints:

• —API_BASE_URL and API_KEY are used by the OpenAI client for judge-proxy LLM calls.
• —LOCAL_ENV_URL points to the local OpenEnv FastAPI server, and defaults to http://localhost:7860.

Optional model override:

• —MODEL_NAME controls the model label printed in structured logs.

Example:

export API_BASE_URL="https://your-judge-proxy.example.com/v1"
export API_KEY="your_key_here"
export LOCAL_ENV_URL="http://localhost:7860"
export MODEL_NAME="gpt-4.1-mini"

Docker Quick Start

1. 1.Build the image:

docker build -t audit-gym .

1. 1.Run the container:

docker run -p 7860:7860 audit-gym

1. 1.Verify the API is up:

curl http://localhost:7860/tasks

If port 7860 is already in use, map a different host port:

docker run -p 7861:7860 audit-gym

Task Suite

Action Space

The environment accepts actions as JSON objects.

list_files

• —Purpose: List available files in assets.
• —Required fields: action_type
• —Example:

{
  "action_type": "list_files"
}

read_asset

• —Purpose: Read a specific asset file.
• —Required fields: action_type, target
• —Example:

{
  "action_type": "read_asset",
  "target": "employees.json"
}

submit_answer

• —Purpose: Submit final findings for grading.
• —Required fields: action_type, content
• —Example:

{
  "action_type": "submit_answer",
  "content": "Inactive admins: u004, u005, u007"
}

Observation Space

Each step returns a structured observation object with task and progress context.

• —session_id: Unique run identifier.
• —task_id: Current task key.
• —task_title: Human-readable task name.
• —difficulty: One of easy, medium, hard.
• —step, max_steps: Current progress counters.
• —prompt: Task instruction shown to the agent.
• —available_assets: Files the agent can inspect.
• —accessed_assets: Files already read this episode.
• —findings: Collected findings during the run.
• —report_draft: Current draft answer text.
• —progress: Normalized progress score in [0.0, 1.0].
• —done: Whether the task is complete.
• —status: Current episode status string.
• —recent_events: Recent action outcomes.
• —state_summary: Compact textual summary.

Reward Model

Dense reward shaping is used to encourage audit-like behavior.

• —+0.05 for list_files
• —+0.10 for reading task-relevant assets
• —+0.80 for a correct final answer
• —-0.02 repeated action penalty

Final task grading is normalized to [0.0, 1.0] through task-specific evaluators.

API Endpoints

• —POST /reset: Start or reset a session for a task.
• —POST /step: Execute one action and receive observation + reward.
• —GET /state: Retrieve the current observation for a session.
• —GET /tasks: List available tasks.