CoolFace
Apppublic

Vedi23/internet-diagnosis-env

sourceHugging Facemitupdated 6mo agoView on Hugging Face
0likes
App README

Your README.md should now start like this:

title: Internet Diagnosis Env emoji: ๐ŸŒ colorFrom: blue colorTo: green sdk: docker pinned: false license: mit ---

๐ŸŒ Internet Outage Diagnosis Environment

An OpenEnv environment where an AI agent diagnoses real-world internet connectivity failures.

What is this?

This environment simulates a real-world network support scenario. An AI agent receives network diagnostic data (ping times, packet loss, signal strength, DNS status) and must identify the root cause of internet outages.

This is a task humans actually do every day โ€” network engineers, ISP support staff, and IT helpdesks diagnose these issues constantly.


Observation Space

What the AI sees at each step:

FieldTypeDescription
pingtorouterfloatPing in ms, -1 means timeout
pingtoispfloatPing in ms, -1 means timeout
pingtogooglefloatPing in ms, -1 means timeout
packet_lossfloat0-100%, 0 is perfect
signal_strengthfloatdBm, closer to 0 is better
dns_resolutionbooleanTrue if DNS is working
connected_devicesintegerDevices on network
error_messagestringOptional error details
task_descriptionstringWhat to diagnose

Action Space

What the AI can do:

ActionWhen to use
diagnose_deviceOnly one device affected
diagnose_routerRouter unreachable, all devices fail
diagnose_ispRouter fine but no internet
diagnose_dnsCan ping IPs but websites fail
diagnose_partialMultiple overlapping issues
requestmoreinfoData too ambiguous
escalatetoengineerNeeds physical inspection

Along with:

  • โ€”failing_component โ€” what specifically is broken
  • โ€”suggested_fix โ€” how to fix it
  • โ€”confidence โ€” how sure the AI is (0.0 to 1.0)

Tasks

Task 1 โ€” Easy: Single Point Failure

  • โ€”2 scenarios with one obvious failure
  • โ€”Max 3 steps per episode
  • โ€”Pass score: 0.6

Task 2 โ€” Medium: Multi-Layer Diagnosis

  • โ€”2 scenarios requiring careful analysis
  • โ€”Max 5 steps per episode
  • โ€”Pass score: 0.5

Task 3 โ€” Hard: Complex Intermittent Failure

  • โ€”2 scenarios with overlapping issues
  • โ€”Max 7 steps per episode
  • โ€”Pass score: 0.4

Reward Function

SituationReward
Correct diagnosis+1.0
Right component, wrong category+0.4
Smart request for more info+0.3
Unnecessary escalation-0.1
Wrong diagnosis-0.3
High confidence on correct answer+0.1 bonus

Baseline Scores

Running baseline.py with llama-3.1-8b-instant via Groq:

TaskScore
Task 1 (Easy)1.0
Task 2 (Medium)0.85
Task 3 (Hard)0.15
Average0.67

Setup Instructions

Option 1 โ€” Run Locally

Step 1 โ€” Install dependencies:

bash
pip install -r requirements.txt

Step 2 โ€” Start the server:

bash
python server.py

Step 3 โ€” Run baseline:

bash
export GROQ_API_KEY=your-key-here
python baseline.py

Step 4 โ€” Visit interactive docs:

http://127.0.0.1:7860/docs

Option 2 โ€” Run with Docker

Step 1 โ€” Build:

bash
docker build -t internet-diagnosis-env .

Step 2 โ€” Run:

bash
docker run -p 7860:7860 internet-diagnosis-env

API Endpoints

EndpointMethodDescription
/GETHealth check
/resetPOSTStart new episode
/stepPOSTSubmit diagnosis
/stateGETCurrent state
/tasksGETList all tasks

Project Structure

internet-diagnosis-env/
โ”œโ”€โ”€ models.py        # Pydantic data models
โ”œโ”€โ”€ scenarios.py     # Network failure scenarios
โ”œโ”€โ”€ tasks.py         # Tasks and graders
โ”œโ”€โ”€ environment.py   # Core step/reset/state logic
โ”œโ”€โ”€ server.py        # FastAPI server
โ”œโ”€โ”€ baseline.py      # Baseline inference script
โ”œโ”€โ”€ openenv.yaml     # OpenEnv metadata
โ”œโ”€โ”€ Dockerfile       # Container setup
โ”œโ”€โ”€ requirements.txt # Dependencies
โ””โ”€โ”€ README.md        # This file