snehagupt/news_investigation_agent

title: Fake News Investigation Agent emoji: 📰 colorFrom: blue colorTo: purple sdk: docker app_file: app.py pinned: false ---

Fake News Investigation Agent - OpenEnv Environment

Overview

This is a complete, production-ready OpenEnv-compliant environment for training and evaluating AI agents on fake news detection through multi-step investigation.

Unlike simple ML classifiers, this environment simulates a real-world investigative process where agents must:

• —Analyze article text for red flags
• —Check source credibility
• —Detect bias and one-sided reporting
• —Verify key claims
• —Make informed final decisions

The environment includes sophisticated reward shaping, state management, and deterministic grading to evaluate agent performance across multiple dimensions.

Key Features

✅ Multi-Step Reasoning - Agents must investigate before deciding (not a simple classifier) ✅ Sophisticated Reward Shaping - Incentivizes thorough investigation with correct conclusions ✅ Dynamic State Management - State evolves based on agent actions ✅ Deterministic Grading - Objective evaluation based on correctness, efficiency, and action quality ✅ 3 Difficulty Levels - Easy, medium, and hard investigation tasks ✅ Type-Safe Design - Full Pydantic model validation ✅ OpenEnv Compliant - Follows industry standards for environments ✅ Baseline Agent Included - Demonstrates interaction with environment ✅ Docker Support - Easy containerization and deployment

Environment Architecture

Core Components

openenv_env/
├── env.py              # Main environment class (FakeNewsInvestigationEnv)
├── state.py            # Pydantic models for state/action/reward
├── actions.py          # Action processing and heuristic analysis
├── tasks/
│   └── __init__.py     # Task definitions (3 difficulty levels)
├── graders/
│   └── __init__.py     # Deterministic grading system
└── baseline_agent.py   # Example agent implementation

State Space

The agent observes:

{
    "text": str,                  # News article content
    "source": str,                # Publication/source name
    "analysis_done": bool,        # Text analysis completed?
    "source_checked": bool,       # Source credibility checked?
    "bias_checked": bool,         # Bias detection completed?
    "claims_verified": bool,      # Claims verification completed?
    "steps_taken": int,           # Number of steps taken
    "max_steps": int,             # Maximum allowed steps
    "final_answer": str|None,     # REAL, FAKE, or None
    "action_history": list,       # History of actions
    "internal_evidence": dict     # Evidence gathered
}

Action Space

Six discrete actions available:

Reward System

Key Design Principles:

• —Base rewards for investigation actions (+0.10 to +0.15)
• —Bonuses for finding meaningful evidence
• —Heavy rewards for CORRECT final answers (+0.50)
• —Heavy penalties for WRONG final answers (-0.50)
• —Penalties for repetition (-0.10) and inefficiency (-0.05/step)
• —All rewards normalized to [0.0, 1.0]

Reward Shaping Strategy:

1. 1.Investigation actions provide consistent positive rewards
2. 2.Repeated actions are penalized
3. 3.Final answers dominate overall reward
4. 4.Efficiency bonus for fewer steps
5. 5.Thoroughness bonus for using multiple investigation methods

Grading System

Episodes are graded on 4 dimensions:

1. Correctness (Weight: 70%)

• —1.0 if final answer matches ground truth
• —0.0 if incorrect
• —This is the primary metric

2. Efficiency (Weight: 10%)

• —Based on steps taken vs. optimal (3-4 steps)
• —Formula: 1.0 - ((steps - optimal) / max_steps)
• —Rewards faster conclusions

3. Action Quality (Weight: 10%)

• —Rewards diverse, meaningful actions
• —Penalizes repetition
• —Bonus for using all 4 investigation methods

4. Thoroughness (Weight: 10%)

• —Percentage of investigation methods used
• —Encourages multiple analysis angles
• —Ensures multi-step reasoning

Final Score Formula:

if correct:
    score = 0.5 × correctness + 0.2 × efficiency + 0.15 × quality + 0.15 × thoroughness
else:
    score = 0.7 × correctness + 0.1 × efficiency + 0.1 × quality + 0.1 × thoroughness

Final Score ∈ [0.0, 1.0]

Tasks (3 Difficulty Levels)

Easy Tasks

Example: Obvious Fake with Multiple Red Flags

• —ALL CAPS text
• —Anonymous sources
• —Conspiracy claims
• —No evidence
• —Ground Truth: FAKE

Medium Tasks

Example: Misleading Headlines

• —Correlation presented as causation
• —Small sample sizes
• —Out-of-context truths
• —Technically truthful but misleading
• —Ground Truth: FAKE

Hard Tasks

Example: Realistic Unverified Claims

• —Plausible technology announcements
• —Unverified claims
• —Expert skepticism
• —No independent verification yet
• —Realistic reporting
• —Ground Truth: REAL

Quick Start

Installation

# Clone repository
cd investigation_agent

# Install dependencies
pip install -r requirements.txt

Basic Usage

from openenv_env.env import FakeNewsInvestigationEnv
from openenv_env.state import Action, ActionType

# Create environment
env = FakeNewsInvestigationEnv(max_steps=10)

# Reset with a task
state = env.reset(task_id="easy_001")

# Take an action
action = Action(action_type=ActionType.ANALYZE_TEXT)
state, reward, done, info = env.step(action)

# Continue investigation...
action = Action(action_type=ActionType.CHECK_SOURCE)
state, reward, done, info = env.step(action)

# Make final decision
action = Action(action_type=ActionType.FINAL_FAKE)
state, reward, done, info = env.step(action)

# Grade the episode
grade = env.grade_episode()
print(f"Final Score: {grade.final_score}")

Run Baseline Agent

# Run baseline agent on all tasks
python -m openenv_env.baseline_agent

Expected Output:

======================================================================
INVESTIGATION TASK: easy_001
Difficulty: easy
Article Source: TruthCentral.EXPOSED
======================================================================

Step 1: analyze_text          → Reward: +0.10
    - Sensationalism score: 0.75
    - Emotional language: True
    - ALL CAPS usage: True

Step 2: check_source          → Reward: +0.15
    - Credibility: 0.15
    - Known unreliable: True

Step 3: detect_bias           → Reward: +0.12
    - One-sided reporting: True

Step 4: verify_claim          → Reward: +0.12
    - Unverifiable claims: 2

Step 5: final_fake            → Reward: +0.50
    - Correct!

======================================================================
INVESTIGATION RESULTS
======================================================================
Final Score: 0.89 / 1.0

Docker Usage

Build Image

docker build -t fake-news-agent:latest .

Run Container

docker run --rm fake-news-agent:latest

Run with Volume Mounting

docker run --rm -v $(pwd)/logs:/app/logs fake-news-agent:latest

OpenEnv Compliance

This environment fully complies with OpenEnv standards:

✅ Typed Models - All state/action/reward models use Pydantic ✅ Standard Interface - reset(), step(), state() functions ✅ Observation Space - Well-defined state with clear semantics ✅ Action Space - Discrete actions with clear descriptions ✅ Reward Space - Continuous rewards in [0.0, 1.0] ✅ Grading Function - Deterministic episode evaluation ✅ Configuration File - Complete openenv.yaml specification ✅ Documentation - Comprehensive README and inline comments

See openenv.yaml for full specification.

File Structure

investigation_agent/
├── openenv_env/
│   ├── __init__.py
│   ├── env.py                 # Main environment
│   ├── state.py               # Pydantic models
│   ├── actions.py             # Action processing
│   ├── baseline_agent.py      # Example agent
│   ├── openenv_config.py      # OpenEnv specification
│   ├── tasks/
│   │   └── __init__.py        # Task definitions
│   └── graders/
│       └── __init__.py        # Grading system
├── openenv.yaml               # Configuration file
├── requirements.txt           # Python dependencies
├── Dockerfile                 # Container setup
└── README.md                  # This file

Implementation Details

Why This Is Not a Simple Classifier

1. 1.Multi-Step Reasoning - Agent must gather evidence before deciding
2. 2.State Evolution - State changes based on investigation actions
3. 3.Dynamic Rewards - Rewards vary based on timing and thoroughness
4. 4.Sequential Decision Making - Action N depends on state at step N-1
5. 5.Efficiency vs Accuracy Tradeoffs - Agent must balance investigation depth with step efficiency

Reward Shaping Rationale

The reward system is carefully shaped to:

1. 1.Encourage Investigation - Base rewards for each investigation action
2. 2.Discourage Shortcuts - Heavy penalty for making final call without investigation
3. 3.Reward Correctness - Dominant factor in final score
4. 4.Incentivize Efficiency - Bonus for using optimal number of steps
5. 5.Promote Thoroughness - Bonus for using diverse investigation methods
6. 6.Prevent Exploitation - Penalties for repetition and inefficiency

Advancing Agent Development

Tips for Building Better Agents

1. 1.Observe Evidence - Use state.internal_evidence to understand findings
2. 2.Chain Investigations - Use results from one action to guide the next
3. 3.Balance Thoroughness - Don't waste steps on repeated investigations
4. 4.Learn Task Patterns - Different tasks may require different strategies
5. 5.Track Efficiency - Monitor steps vs. reward tradeoff

Extending the Environment

1. 1.Add Custom Tasks - Edit tasks/__init__.py
2. 2.Modify Reward Function - Adjust env.py reward calculations
3. 3.Add New Analysis Methods - Extend actions.py with new processors
4. 4.Implement Learning Agent - Use baseline as template

Testing & Validation

# Run all tests
pytest

# Run specific test
pytest openenv_env/test_env.py::test_step_function

# Check code quality
flake8 openenv_env/
mypy openenv_env/

# Check test coverage
pytest --cov=openenv_env

Performance Benchmarks

With the included baseline agent:

Key Insights

1. 1.Multi-step reasoning is crucial - Agents that investigate thoroughly score higher
2. 2.Efficiency matters - But not at the cost of accuracy
3. 3.Diversity in investigation - Using multiple methods improves scores
4. 4.Source credibility - Often the strongest signal for fake news
5. 5.Pattern matching - Sensationalism and emotional language are reliable indicators

References

• —OpenEnv Specification: openenv.yaml
• —Baseline Agent: baseline_agent.py
• —Environment Source: env.py
• —State Models: state.py

License

This project is provided as-is for research and educational purposes.

Support

For issues or questions, refer to:

• —openenv.yaml for environment specification
• —openenv_env/env.py for implementation details
• —openenv_env/baseline_agent.py for usage examples
• —Inline code comments for technical details

Build Date: April 2026 Version: 1.0.0 Status: Production Ready ✅