AryanJain7031/dead-internet-detective

Dead Internet Detective

We trained an LLM to investigate disinformation the way a journalist would — not by reading and classifying, but by following leads through a synthetic internet we built from scratch.

    

The Problem with Disinformation Detection

Most approaches treat disinformation as a classification problem. Feed in an article, get back a label. True or false.

This misses the point entirely.

Sophisticated disinformation doesn't fail because it sounds suspicious. It fails because its citations don't trace back to anything real. Because the author's name appears on forty different sites with conflicting credentials. Because the domain was registered two weeks before the story broke. Because the "study" being cited was quietly retracted. None of that is in the headline. You have to investigate.

The gap between classifying text and investigating a claim is enormous — and almost nobody has tried to close it with RL training on LLMs. That's what this project does.

What We Built

Dead Internet Detective is a multi-step investigation environment. The agent receives a claim and a set of dossier URLs — entry points into a fully synthetic internet we built specifically for this project.

That synthetic internet is the core of what makes this environment interesting. It's not a static dataset. Every episode generates a fresh set of pages: legitimate-looking credible sources, SEO farms recycling the same talking points, wellness blogs with no real author credentials, and — at hard difficulty — high-quality fakes that convincingly spoof real credible domains. The citation chains look plausible. They just don't lead anywhere real.

The agent has 12 investigative tools to work through the evidence:

The agent maintains a case file throughout the investigation — a running record of confirmed facts, disputed claims, flagged synthetic sources, contradictions found, and overall confidence. When it's ready (or when it runs out of steps), it files a report: verdict, confidence score, evidence chain, and reasoning.

Why This Needed RL

You can't train an investigator with supervised learning on verdict labels. The investigation process matters as much as the conclusion. An agent that guesses correctly by luck and uses zero tools should score lower than one that builds a real evidence chain — even if both get the right verdict.

The reward function captures this:

A lazy agent that reads one page and guesses "true" gets partial credit on verdict accuracy. It gets zero on everything else. The only path to a high total reward is a genuine investigation. We also penalize hitting the step limit — an agent that spins out without filing a report loses an additional 0.15.

This reward function is genuinely hard to game. The environment was designed so that no single tool call pattern can systematically exploit it.

Difficulty Levels

Training

We used GRPO (Group Relative Policy Optimization) via TRL + Unsloth to train Llama-3.1-8B-Instruct across three progressive phases. The progression was intentional: each phase expanded both the difficulty and the expected behavior.

Phase 1 teaches the agent that tools exist and should be used. Phase 2 forces it to handle cases where the evidence is genuinely mixed and a single visit isn't enough. Phase 3 refines efficiency — getting to the right answer without burning the step budget on redundant calls.

Results

Reward Curve (Phase 1 · easy difficulty · ~500 steps)

[image]

Training step on x-axis, episode reward on y-axis. Red line = rolling mean. The agent begins near the untrained baseline (~0.15) and climbs toward ~1.45 by step 500. GRPO on Llama-3.1-8B-Instruct.

Before vs. After: What Actually Changed

[image]

Each component scored separately. Gray = untrained baseline (visits dossier URLs, guesses "true"). Red = trained agent after Phase 1. The trained agent uses citation tracing, author lookups, and cross-referencing — behaviors the untrained model never exhibited.

The untrained agent visits URLs and guesses. The trained agent traces citations, checks author credibility across domains, identifies contradictions between sources, and builds an evidence chain before filing. That's not a marginal improvement — it's a qualitatively different kind of behavior.

Architecture

┌─────────────────────────────────────────────────────────┐
│                        Colab / Local                    │
│                                                         │
│   ┌──────────────────┐       ┌─────────────────────┐   │
│   │  LLM Agent       │──────▶│  DeadInternetClient │   │
│   │  (Llama-3.1-8B)  │◀──────│  (client/client.py) │   │
│   └──────────────────┘       └────────┬────────────┘   │
└────────────────────────────────────────│────────────────┘
                                         │ HTTP (REST)
                          ┌──────────────▼──────────────┐
                          │    FastAPI Server            │
                          │    /reset  /step  /state     │
                          └──────────────┬───────────────┘
                                         │
                          ┌──────────────▼───────────────┐
                          │  DeadInternetEnvironment     │
                          │  12 Tools · 5 Graders        │
                          │  Synthetic Internet          │
                          └──────────────────────────────┘

Quick Start

Against the live HF Space (no local setup required):

import requests

BASE = "https://aryanjain7031-dead-internet-detective.hf.space"

# Start a new investigation
r = requests.post(f"{BASE}/reset", json={"task_id": "easy", "seed": 42})
session_id = r.json()["session_id"]
obs = r.json()["observation"]

print("Claim:", obs["claim"])
print("Dossier:", obs["dossier_urls"])

# Follow a lead
r2 = requests.post(f"{BASE}/step", json={
    "session_id": session_id,
    "action": {"tool": "visit_page", "params": {"url": obs["dossier_urls"][0]}}
})
print("Reward:", r2.json()["reward"])
print("Done:", r2.json()["done"])

Run the trained P3 model against the live environment:

Use the Eval Space — select P3, pick difficulties, hit Run. No GPU required.

Or locally (requires CUDA):

python evaluate.py --model p3 --vs-baseline --difficulties easy medium hard --n-episodes 3

Local setup:

git clone https://github.com/aj7075/dead-internet-detective.git
cd dead-internet-detective && pip install -r requirements.txt
uvicorn dead_internet_detective.server.app:app --reload --port 8000

Links

Built with [OpenEnv](https://github.com/huggingface/open-env) · [TRL/GRPO](https://github.com/huggingface/trl) · [Unsloth](https://github.com/unslothai/unsloth)