Pankei/soc-narrative-grpo32-qwen3-14b

SOC Narrative GRPO — Qwen3-14B (Step 32, Budget 512)

  

LoRA adapter for Qwen/Qwen3-14B trained with GRPO LoRA on 512 balanced user/day windows (CERT R4.2) (step 32).

Description

SOC Narrative is a framework for insider threat detection using small open-weight LLMs. A model receives a user/day window of events from the CERT Insider Threat Dataset R4.2 and must produce a structured response with:

• —Risk label: normal, suspicious, or malicious
• —Evidence: cited event IDs supporting the decision
• —Reasoning: brief explanation of the investigation logic

This project explores whether small LLMs (3B–14B) can match or exceed traditional ML baselines for UEBA (User and Entity Behavior Analytics).

Metrics (devbalanced50)

Quick Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = "Qwen/Qwen3-14B"
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(model, "Pankei/soc-narrative-grpo32-qwen3-14b")

tokenizer = AutoTokenizer.from_pretrained(base)
inputs = tokenizer("<your prompt>", return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(output[0]))

Note: This is a LoRA adapter (~30–160 MB). You need the full base model (Qwen/Qwen3-14B) to load it.

Training Details

• —Base model: Qwen/Qwen3-14B
• —Method: GRPO LoRA
• —Train data: 512 balanced user/day windows (CERT R4.2)
• —Checkpoint: step 32
• —LoRA rank: 32, alpha: 64, target modules: qproj, kproj, vproj, oproj
• —Format: Structured SOC Narrative (risk + evidence + reasoning)
• —Hardware: NVIDIA H100 (80 GB)

Limitations

• —Evaluated on a small balanced sample (n=50) — results may not generalize to production distributions
• —GRPO improves over SFT in accuracy (+8pp), format compliance (+20pp), and actionability (+12pp), at a modest recall cost vs SFT.
• —Dataset is based on synthetic insider threat scenarios from CERT R4.2 (2016) — real-world performance may differ

Citation

@misc{soc-narrative-2026,
  author = {Research project},
  title = {SOC Narrative: Small LLMs for UEBA / Insider Threat Detection},
  year = {2026},
  howpublished = {\url{https://github.com/Pancake2021/research_work_by_a_student}}
}