CoolFace
Apppublic

Lishika/crime-investigation-rl-env

sourceHugging Faceupdated 5mo agoView on Hugging Face
0likes
App README

Crime Investigation RL Environment

A multi-agent reinforcement learning environment where LLM detectives learn to solve mysteries through strategic interrogation and lie detection.

Problem

This environment trains a critical capability gap in modern LLMs: reasoning under adversarial deception. Agents must maintain internal state across multi-turn interrogations, triangulate contradictory information, and make grounded, evidence-based accusations instead of jumping to conclusions.

Environment Design

  • —Observation: The agent sees a case briefing, their previous conversation history, and revealed physical evidence. Ground truth is strictly hidden.
  • —Actions:
  • —ASK <name>: <question>
  • —EVIDENCE <evidence_name>
  • —ACCUSE <suspect_name>
  • —NPCs: Suspects and witnesses are deterministic, hashing-based agents. Guilty suspects lie logically, and witnesses offer biased fragments. No LLM is in the reward path.
  • —Reward Rubrics:
  • —R1 (Accusation): +12 base, +2 evidence bonus, -5 premature penalty, -8 wrong.
  • —R2 (Contradiction): +1.5 for catching a lie.
  • —R3 (Evidence): +0.5 for valid evidence, -0.4 for hallucinated evidence.
  • —R4 (Consistency): -5.0 anti-hacking penalty for self-contradiction.
  • —R5 (Efficiency): -0.3 time tax per step.
  • —R6 (Strategy): +0.5 new topics, -0.5 redundant questions.
  • —R7 (First Try): +2.0 bonus.

Results

[image] [image]

The baseline model initially struggles, frequently guessing prematurely or hallucinating evidence. After GRPO training on the composable reward rubrics, the agent learns to methodically gather evidence, cross-reference witness statements, and achieve a significantly higher accusation accuracy with fewer turns.

Quick Start

bash
pip install openenv-core
pip install git+https://huggingface.co/spaces/Lishika/crime-rl-env
python
from client.client import CrimeEnvClient

client = CrimeEnvClient("http://localhost:7860")
obs = client.reset()

print(obs["briefing"])
result = client.step("ASK Yusuf Okafor: Where were you?")
print(result["reward"])

Links

  • —HuggingFace Space: HuggingFace Space Demo
  • —Mini-blog: mini_blog.md
  • —Training script: training/train_local.py
Lishika/crime-investigation-rl-env · CoolFace