Abiray/Shieldstral-1.0-3B-GGUF

Shieldstral 1.0 3B - GGUF

This repository contains GGUF format quantized weights for Mistral AI's Shieldstral-1.0-3B.

Shieldstral is a compact, policy-adaptive multimodal safety classifier. Instead of relying on a fixed set of moderation categories, it evaluates content against a safety policy expressed in natural language. It is ideal for lightweight, real-time content moderation on edge devices or standard consumer hardware.

These quantized GGUF models are optimized for CPU and GPU inference using llama.cpp and support both text-only and multimodal (text + image) moderation.

Available Files

1. Language Model (Text Weights)

Choose the quantization level that best fits your RAM and speed requirements. Q4_K_M is highly recommended for a great balance of speed and accuracy.

2. Multimodal Projector (Vision Weights)

To use the model for image moderation, you must download one of the mmproj files. The vision projector cannot be heavily quantized without destroying image recognition capabilities, so they are kept in full 16-bit precision.

Note: You only need one projector file, and it can be paired with any of the quantized text models above.

How to Use with llama.cpp

Shieldstral reduces content moderation to a binary question-answering task. Ensure you wrap your moderation question in [INST] ... [/INST] tags.

1. Text-Only Moderation

If you only need to moderate text, you only need to download a text GGUF (e.g., Shieldstral-1.0-3B-Q4_K_M.gguf).

./llama-cli -m Shieldstral-1.0-3B-Q4_K_M.gguf \
  -n 128 \
  -c 1024 \
  -p "[INST] Evaluate this text for harmful content: 'I love programming in Python.' [/INST]"

2. Multimodal Moderation (Image + Text)

To evaluate an image, you must download both a text model and an mmproj model. You must also increase the context size (-c) to accommodate the image tokens (4096 is recommended).

./llama-cli -m Shieldstral-1.0-3B-Q4_K_M.gguf \
  --mmproj mmproj-Shieldstral-1.0-3b-BF16.gguf \
  --image /path/to/your/image.jpg \
  -n 128 \
  -c 4096 \
  -p "[INST] Evaluate this image for harmful or violent content. [/INST]"

Limitations & Considerations

• —Shieldstral outputs a continuous confidence score based on the probability of a "yes" or "no" token. In terminal interfaces, you will simply see the text "yes" or "no".
• —Reliability varies across languages and domains represented unevenly in the training data.
• —Check the original Mistral AI model card for complete documentation, prompt-engineering tips, and ethical considerations.

License

This model is released under the Apache 2.0 License, permitting both commercial and non-commercial open-weights use.