FrameByFrame/llm-defence-scanner-lfm2.5-1.2b

LLM Defence Scanner — LFM2.5 1.2B


A fine-tuned LFM2.5-1.2B-Instruct model for AI guardrail classification. Single LLM, six categories, structured JSON verdicts. Works for input guard (scan user prompts before they reach an LLM) and output guard (scan LLM responses before they reach the user).

 — runnable end-to-end with all six categories.

Capabilities

Categories are scoped per request via applied_policies — different tenants enable different scanners with different parameters.

Benchmarks

Held-out test set, 841 records, English only. Model output is post-processed deterministically (apply_policy_postprocess) for code allow-list and competitor list filtering.

Per-category matched-accuracy:

Inference

End-to-end usage — model load, prompt rendering, post-processor, six category examples for both input-leg and output-leg — is in [`release_v1_demo.ipynb`](https://huggingface.co/FrameByFrame/llm-defence-scanner-lfm2.5-1.2b/blob/main/release_v1_demo.ipynb). Download and run in Jupyter or Colab.

For production serving with vLLM:

vllm serve FrameByFrame/llm-defence-scanner-lfm2.5-1.2b \
  --served-model-name llm-defence-scanner \
  --max-model-len 4096 \
  --gpu-memory-utilization 0.85 \
  --no-enable-prefix-caching

p50 latency ~360 ms on RTX PRO 6000 Blackwell, p95 ~870 ms.

Output schema

{
  "overall_blocked": true,
  "severity": "high",
  "language": {"dominant": "en", "alternates": [], "script": "latin", "code_mixed": false},
  "scenario": {"name": "banking", "profile": "retail_banking_kyc"},
  "categories": [
    {
      "name": "pii",
      "matched": true,
      "matches": [
        {"text": "alice.tan@example.com", "kind": "email"},
        {"text": "+1-415-555-2244", "kind": "phone"}
      ]
    }
  ],
  "reason": "The input contains personally identifiable information."
}

Training

• —Base: LiquidAI/LFM2.5-1.2B-Instruct
• —Method: LoRA (r=16, α=16) on q/k/v/o + w1/w2/w3, then merged
• —Dataset (v5): ~11k records spanning synthetic multi-category inputs, ai4privacy/pii-masking-65k PII spans, bantopics safety pairs, embedded prompt-injection synthesis. English only, 8-gram leakage-filtered against the test split.
• —Hyperparameters: effective batch=32, LR=2e-4, 2 epochs
• —Hardware: single RTX PRO 6000 Blackwell (96GB)
• —Final eval_loss: 0.0015

Citation

@misc{mariappan2026llmdefence,
  author    = {Mariappan, Vijayachandran},
  title     = {LLM Defence Scanner — LFM2.5 1.2B},
  year      = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/FrameByFrame/llm-defence-scanner-lfm2.5-1.2b}}
}

License

Based on LFM2.5 — subject to the LFM Open License v1.0.