Neethish05/meta-pytorch-hackathon

AI Code Review Arena - OpenEnv Compliant Environment

Why This Problem Matters (Real-World Impact)

Code review is a critical bottleneck in software development. According to industry research:

• —60% of security vulnerabilities are introduced during code development but missed in review
• —$4.61 million is the average cost of a data breach caused by missed vulnerabilities
• —73% of developers say code review quality directly impacts project success
• —AI code review tools have a false positive rate of 40%, leading to alert fatigue

Current AI code evaluation systems fail because they:

1. 1.Use toy examples that don't reflect real-world complexity
2. 2.Don't test multi-step reasoning across multiple files
3. 3.Ignore deceptive commit messages that mislead human reviewers
4. 4.Lack structured evaluation that rewards intelligent behavior

AI Code Review Arena fills this gap by providing a realistic environment that tests AI's ability to detect security vulnerabilities, logic errors, and quality issues in authentic Pull Request scenarios.

Action Space (Strict Schema)

Machine-Readable Action Structure

{
  "type": "comment | approve | request_changes | flag_security | suggest_optimization",
  "line": 12,
  "file": "auth.py", 
  "message": "Potential SQL injection vulnerability due to unparameterized query",
  "confidence": 0.8,
  "issue_type": "security | critical | logical | performance | quality"
}

Action Type Requirements

Validation Rules

• —Line numbers must be 1-based integers
• —File names must match files in the diff
• —Confidence must be 0.0-1.0
• —Issue type required for specific actions
• —No ambiguity - each action has precise requirements

Observation Schema

Complete Observation Structure

{
  "diff": "unified diff of all changes",
  "files": ["auth.py", "database.py", "utils.py"],
  "commit_message": "Enhanced security with ORM integration",
  "history": [
    {
      "step": 1,
      "action": "flag_security",
      "line": 15,
      "message": "SQL injection vulnerability detected",
      "reward": 0.5
    }
  ],
  "step_count": 3,
  "current_issues": [
    {
      "type": "security",
      "line": 15,
      "file": "database.py",
      "description": "SQL injection in user query",
      "severity": 1.0,
      "detected": false
    }
  ],
  "echoed_message": "Action: flag_security, Message: SQL injection vulnerability detected"
}

Observation Components

• —diff: Complete unified diff showing all code changes
• —files: List of modified file names
• —commit_message: May be deceptive (misleading descriptions)
• —history: Previous actions with rewards
• —step_count: Current step in episode
• —current_issues: Hidden issues (not visible to agent, used for scoring)
• —echoed_message: Last action echoed back

Reward Logic (Explicit & Deterministic)

Base Reward Calculations

IF correct issue detected:
    +0.3 (base reward)

IF critical issue detected:
    +0.5 (critical bonus)

IF explanation contains relevant keywords:
    +0.2 (keyword bonus)

IF high confidence (>=0.8) on correct issue:
    +0.1 (confidence bonus)

IF false positive (issue doesn't exist):
    -0.2 (false alarm penalty)

IF approve when critical issue exists:
    -0.7 (heavy penalty)

IF repeated identical actions:
    -0.3 (spam penalty)

IF generic comments ("looks good", "seems fine"):
    -0.2 (low-effort penalty)

IF too many steps (>10):
    -0.05 per extra step (efficiency penalty)

Deception Scoring

IF resisted deceptive commit message:
    +0.2 (deception resistance bonus)

IF fell for deception:
    -0.2 (deception penalty)

IF consistently resists deception:
    +0.15 (consistency bonus)

Confidence Scoring

IF correct + high confidence (>=0.8):
    +0.1 (confidence bonus)

IF wrong + high confidence (>=0.8):
    -0.1 (overconfidence penalty)

IF correct + low confidence (<=0.3):
    -0.05 (underconfidence penalty)

Total Reward Formula

final_reward = base_reward + confidence_bonus + keyword_bonus 
             + deception_bonus - penalties

All rewards are deterministic - no randomness, same action always produces same reward.

Task Difficulty Explanation

Easy Task - Array Index Bug (Success Threshold: 0.7)

Objective: Detect obvious index out of bounds error

Scenario:

# Before
for i in range(len(arr)):
    print(arr[i])

# After  
for i in range(len(arr)):
    print(arr[i+1])  # Bug: index out of bounds

Difficulty Factors:

• —Single file, obvious bug
• —Clear error pattern
• —High success threshold (0.7)
• —Max 10 steps

What AI Must Do:

• —Identify arr[i+1] exceeds array bounds
• —Use comment action with correct line number
• —Provide specific error description

Medium Task - Authentication Logic Flaw (Success Threshold: 0.6)

Objective: Find hidden authentication issue with misleading commit message

Scenario:

# Commit: "Added admin check for security" (misleading)

# Before
if user.is_authenticated and user.is_admin:
    return True

# After
if user.is_admin:  # Bug: missing authentication check
    return True

Difficulty Factors:

• —Misleading commit message
• —Security issue requires careful analysis
• —Medium success threshold (0.6)
• —Max 10 steps

What AI Must Do:

• —Ignore misleading commit message
• —Detect missing authentication
• —Use flag_security action
• —Resist deception for bonus points

Hard Task - Advanced Multi-Layer SQL Injection (Success Threshold: 0.5)

Objective: Identify SQL injection across multiple files with deception

Multi-File Scenario:

File 1: utils.py (appears safe)

def validate_user_id(user_id: str) -> Optional[int]:
    if not user_id.isdigit():
        return None
    return int(user_id)  # Looks safe but has edge cases

File 2: controllers.py (hidden vulnerability)

def get_user_profile(self, user_id: str, username: str = None):
    validated_id = validate_user_id(user_id)  # "Safe" validation
    # VULNERABILITY: Raw SQL despite validation
    query = f"SELECT * FROM users WHERE id = {validated_id}"
    return self.session.execute(query).fetchone()

Commit Message: "Enhanced security with ORM integration and input validation"

Difficulty Factors:

• —Multi-step dependency: Must analyze utils.py + controllers.py
• —Deceptive validation: Input validation creates false security
• —Mixed ORM/raw SQL: Confusing pattern
• —Misleading commit: Claims security improvements
• —Low success threshold (0.5) - acknowledges difficulty
• —Max 15 steps - allows for complex reasoning

What AI Must Do:

1. 1.Analyze validation function for edge cases
2. 2.Trace data flow from validation to SQL query
3. 3.Understand validation doesn't prevent SQL injection
4. 4.Identify raw SQL usage despite ORM presence
5. 5.Use flag_security with high confidence
6. 6.Resist multiple deception layers

Advanced Features (Competitive Advantages)

1. Deception Score Tracking

• —Tracks AI's ability to resist misleading commit messages
• —Bonus for consistent deception resistance
• —Unique feature most competitors won't have

2. Failure Trap System

• —Prevents spam and random guessing
• —Penalties for repetitive actions
• —Forces intelligent behavior

3. Confidence Scoring

• —Rewards appropriate confidence levels
• —Penalizes overconfidence on wrong issues
• —Encourages calibrated uncertainty

4. Multi-Step Dependencies

• —Issues require analyzing multiple files
• —Tests true reasoning, not pattern matching
• —Reflects real-world code review complexity

Real-World Utility

This environment addresses genuine needs in the AI/ML community:

For Researchers

• —Benchmark Development: Compare different models on realistic code review tasks
• —Multi-Step Reasoning: Study complex reasoning patterns
• —Deception Resistance: Evaluate AI robustness against misleading information

For Industry

• —Agent Evaluation: Test code review AI before deployment
• —Training Data: Generate realistic scenarios for AI training
• —Risk Assessment: Evaluate AI's ability to catch security vulnerabilities

For the OpenEnv Community

• —Novel Domain: Code review is underrepresented in RL environments
• —Structured Evaluation: Precise, reproducible scoring system
• —Real-World Relevance: Direct mapping to software development workflows

Environment Design Principles

Clean State Management

• —Deterministic Reset: Each episode starts with identical state
• —Clear Episode Boundaries: Defined start/end conditions
• —State Consistency: No hidden state between episodes

Sensible Action Spaces

• —Structured Actions: No ambiguity, machine-readable format
• —Realistic Constraints: Actions mirror real code review behavior
• —Type Safety: All actions validated against schema

Useful Reward Shaping

• —Partial Progress: Rewards for incremental improvements
• —Immediate Feedback: No delayed rewards
• —Balanced Incentives: Encourages thorough, intelligent review

Deterministic Evaluation

• —Reproducible Results: Same actions always produce same rewards
• —No Randomness: Pure skill evaluation
• —Transparent Logic: Clear reward calculation rules

Setup and Usage

Installation

# Clone repository
git clone <repository-url>
cd ai-code-review-arena

# Install dependencies
pip install -r requirements.txt

# Install OpenEnv CLI tool
pip install openenv-core

Validation

# Validate OpenEnv compliance
openenv validate

# Run comprehensive validation
python validate.py

Running the Environment

Local Development

# Start Flask API server
python openenv_app.py

# Test reset endpoint
curl -X POST http://localhost:8000/reset \
  -H "Content-Type: application/json" \
  -d '{"task_id": 0}'

Baseline Evaluation

# Set environment variables
export OPENAI_API_KEY="your-api-key"
export MODEL_NAME="gpt-4"
export API_BASE_URL="https://api.openai.com/v1"

# Run baseline inference
python inference.py

Docker Deployment

# Build and run
docker build -t ai-code-review-arena .
docker run -p 8000:8000 ai-code-review-arena

Performance Metrics

The environment tracks comprehensive metrics:

Detection Metrics

• —Issue Detection Rate: Issues found vs total issues
• —Critical Detection Rate: Security/critical issues found
• —False Positive Rate: Incorrect issue identification

Behavior Metrics

• —Deception Resistance: Ability to resist misleading commits
• —Confidence Calibration: Appropriateness of confidence levels
• —Efficiency: Steps taken vs issues found

Scoring Metrics

• —Normalized Score: 0-1 scale for easy comparison
• —Partial Progress: Reward for incremental improvements
• —Consistency: Performance across multiple episodes

Why This Submission Stands Out

1. Real-World Relevance (30% weight)

• —Models actual code review workflow used by thousands of developers
• —Addresses multi-billion dollar software security problem
• —Direct application to AI-assisted development tools

2. Task Quality (25% weight)

• —3 distinct difficulty levels with clear progression
• —Deterministic graders producing 0.0-1.0 scores
• —Hard task requires genuine multi-step reasoning
• —Deception elements test robustness

3. Environment Design (20% weight)

• —Clean state management with proper reset/episode boundaries
• —Strict action schema with no ambiguity
• —Meaningful reward shaping with partial progress signals
• —Failure trap system prevents gaming the environment

4. Code Quality (15% weight)

• —Full OpenEnv compliance with typed models
• —Comprehensive validation passes all checks
• —Docker + HF Spaces ready for deployment
• —Baseline inference with reproducible scores

5. Creativity (10% weight)

• —Deception Score Tracking - unique feature
• —Confidence Scoring - encourages calibrated AI
• —Multi-Step Dependencies - tests true reasoning
• —Failure Trap System - prevents spam behavior

License

MIT License - feel free to use for research and development.

Citation

If you use this environment in your research, please cite:

AI Code Review Arena: An OpenEnv Environment for Realistic Code Review Evaluation
Authors: Code Review Arena Team
Environment: OpenEnv Compliant
Domain: Software Engineering / Security

This environment represents a significant advancement in AI evaluation for code review tasks, combining real-world relevance with rigorous evaluation methodology.