Thorge-AI/llama-guard-3-1b-q8_0.gguf

Llama-Guard-3-1B-GGUF-INT8 (Optimized for Android)

This repository provides a quantized INT8 (Q8_0) version of Meta's Llama-Guard-3-1B, converted to the GGUF format for efficient on-device inference.

Built with Llama

Llama Guard 3-1B is a fine-tuned Llama-3.2-1B pretrained model for content safety classification. This specific GGUF version is a derivative work optimized for mobile and edge deployments where accuracy and memory efficiency are critical.

Why INT8 (Q8_0) instead of INT4?

While Meta provides an official 4-bit version, smaller models (1B) can experience a significant increase in False Positive Rates (FPR) when heavily quantized. For example, in German, the INT4 version's FPR can rise to ~14.5%, leading to excessive over-blocking.

This INT8 version maintains approximately 99% of the original model's precision, making it the "Sweet Spot" for developers who need reliable safety layers on Android without frustrating users through false alarms.

Model Details

• —Base Model: meta-llama/Llama-Guard-3-1B
• —Quantization: INT8 (Q8_0) via llama.cpp
• —File Size: ~1.6 GB
• —Intended Use: Input/Output moderation for LLM-based applications.

Hazard Taxonomy & Policy

The model is trained to detect 13 categories of hazards based on the MLCommons taxonomy:

Usage

Prompt Template

To ensure correct classification, you must use the official Llama-Guard template:

<|begin_of_text|><|start_header_id|>user<|end_header_id|>

[YOUR_INPUT_HERE]<|eot_id|><|start_header_id|>assistant<|end_header_id|>

License & Terms

This model is licensed under the Llama 3.2 Community License Agreement.

Attribution

"Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved."

Acceptable Use Policy

You must comply with Meta's Acceptable Use Policy. By using this model, you agree not to use it for illegal activities, harassment, or any purposes prohibited by Meta's guidelines.

Citation

If you use this model, please cite the original Llama 3 family:

@misc{metallamaguard3,
  author = {Llama Team, AI @ Meta},
  title = {The Llama 3 Family of Models},
  howpublished = {\url{[https://github.com/meta-llama/PurpleLlama/blob/main/Llama-Guard3/1B/MODEL_CARD.md](https://github.com/meta-llama/PurpleLlama/blob/main/Llama-Guard3/1B/MODEL_CARD.md)}},
  year = {2024}
}

Quantization performed by: [Thorge Mrowinski]

Format: GGUF (Compatible with llama-android, llama.rn, and llama.cpp)