Shiggii/qwen-incident-response-grpo

Qwen2.5-0.5B Fine-tuned for Incident Response with Authority Bias Resistance

Fine-tuned for the Meta x Hugging Face OpenEnv Hackathon (April 2026). This adapter is trained to reduce authority bias in incident triage: the agent should prioritize runbook evidence and causal logs over contradictory social pressure in Slack-style messages.

Model Details

• —Base Model: Qwen/Qwen2.5-0.5B-Instruct
• —Adapter: QLoRA / LoRA adapter (Shiggii/qwen-incident-response-grpo)
• —Training Framework: TRL 1.2.0, GRPO
• —Training Steps: 384 optimizer steps
• —Compute: Kaggle free tier (Tesla T4), <10 GPU-hours total
• —Hackathon Context: OpenEnv environment-focused benchmark development

Training Philosophy

We optimized for iteration speed and environment quality over brute-force model scale:

• —Fast prototyping loop for reward/environment design
• —Lightweight fine-tuning footprint suitable for commodity hardware
• —Reproducible setup with public environment + notebook + artifacts

This was the final training iteration after API-based prototyping with llama-3.1-8b-instant to validate task and reward design.

Environment and Task

Environment: https://huggingface.co/spaces/Shiggii/incident-response-detective

The model acts in a synthetic but realistic incident-response environment where each step includes:

• —Logs (timestamped system events)
• —Chat history (social signal, can be adversarial)
• —Runbook (official procedure constraints)

Action Space

The model selects one remediation action from:

• —rollback_deployment
• —scale_infrastructure
• —flush_redis_cache
• —notify_cto
• —restart_api_gateway
• —rotate_db_credentials
• —enable_circuit_breaker
• —purge_cdn_cache

Reward Logic (Environment Ground Truth)

Reward is shaped around:

• —Safety (runbook compliance / avoiding dangerous actions)
• —Efficiency (how quickly the incident is resolved)

Dangerous actions are heavily penalized; correct first-step resolutions receive near-ceiling scores.

Performance

Evaluation during prototyping showed the key behavioral gain on authority-bias scenarios:

Key finding: untrained behavior on Easy adversarial tasks reflects authority bias (following confident chat over runbook/log evidence). After GRPO fine-tuning, the model consistently follows evidence and documented procedures.

Training Curves and Provenance

• —Training/eval metrics are tracked in trainer_state.json (log_history, 384 entries)
• —Aggregate progression (early vs late): 0.946 → 0.995 mean reward
• —Plots are regenerated from authoritative training artifacts via:
• —python regenerate_plots.py

Related artifacts are documented in the environment repo.

Reproduction

• —Training Notebook (Kaggle): https://www.kaggle.com/code/shikharkumarsanjay/notebookb5136cd284

• —Environment (Space): https://huggingface.co/spaces/Shiggii/incident-response-detective

• —Training metrics artifact: trainer_state.json (model repo)

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_id = "Qwen/Qwen2.5-0.5B-Instruct"
adapter_id = "Shiggii/qwen-incident-response-grpo"

tokenizer = AutoTokenizer.from_pretrained(base_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_id,
    torch_dtype=torch.float16 if torch.cuda.is_available() else torch.float32,
    device_map="auto" if torch.cuda.is_available() else None,
)
model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()