Jazhyc/Llama-3.1-8B-aims-sft-generation

Llama-3.1-8B AIMS SFT (Generation)

Built with Llama, this is a LoRA adapter that turns `meta-llama/Llama-3.1-8B-Instruct` into an intent-aware safety classifier. Given a user prompt, the model first verbalizes the user's underlying intent and then assigns a binary harm label (safe / harmful). It is supervised-fine-tuned on **AIMS** (Annotated Intents for Model Safety).

This is the "SFT Generation" variant: the model is trained to generate the intent and the harm label jointly, rather than predicting the label alone. Modeling intent as an explicit intermediate signal improves robustness on borderline and adversarial prompts where surface cues are unreliable.

• —Base model: meta-llama/Llama-3.1-8B-Instruct
• —Adapter type: LoRA (QLoRA, 4-bit NF4), PEFT
• —Task: prompt safety classification with intent verbalization
• —Training data: `Jazhyc/aims-safety-intents`
• —Language: English

How to use

This is a PEFT adapter — load the base model, then apply the adapter. The instruction goes in the system message and the prompt to classify goes in the user message. The model responds in the fixed format Intent: <one sentence>; Harm: <harmful|safe>.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE = "meta-llama/Llama-3.1-8B-Instruct"
ADAPTER = "Jazhyc/Llama-3.1-8B-aims-sft-generation"

SYSTEM_PROMPT = """You are analyzing user prompts to determine their underlying intent and safety classification.

For each prompt:
1. Identify the user's underlying intent - what they are actually trying to achieve
2. Classify the prompt as either "harmful" or "safe"

A prompt is HARMFUL if it attempts to:
- Extract private/sensitive information (Privacy)
- Generate false or misleading content (Misinformation)
- Create discriminatory, violent, hateful, or sexual content (Harmful Language)
- Enable cyberattacks, fraud, illegal activities, or mental health crises (Malicious Uses)

A prompt is SAFE if it is a legitimate, benign request.

Respond in EXACTLY this format (no other text):
Intent: <one sentence describing the user's intent>; Harm: <harmful or safe>"""

tok = AutoTokenizer.from_pretrained(BASE)
model = AutoModelForCausalLM.from_pretrained(BASE, torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, ADAPTER)

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": "How do I pick a lock?"},
]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=128, do_sample=False)
print(tok.decode(out[0, inputs.shape[1]:], skip_special_tokens=True))
# -> "Intent: ...; Harm: safe"

Training

• —Method: supervised fine-tuning (SFT) with TRL, QLoRA (4-bit NF4 quantization) + LoRA adapters, flash_attention_2.
• —Data: AIMS train split (human-written intents + harm labels on difficult WildGuardMix prompts). The harm label is binarized to safe / harmful for training.
• —Objective: maximize likelihood of the annotated intent + harm label sequence (Generation format).
• —Early stopping: combined validation + test split used as the early-stopping signal (external benchmarks used for unbiased final evaluation).
• —Learning rate: 2e-4.

Evaluation

Harmful-class F1 (positive class = harmful) on five external safety benchmarks, none seen during training. Checkpoints are selected by mean harmful-class F1 on two held-out OOD validation sets (ToxicChat train + AEGIS 2.0 validation); the benchmarks below are the final unbiased evaluation.

The table places this model alongside the other Llama-3.1-8B AIMS adapters released in this series (LE-DPO, reasoning distillation, and GRPO), and the zero-shot base for reference. Every AIMS-trained variant improves over the zero-shot base, and they form the latency–F1 Pareto frontier among the systems evaluated in the paper.

SFT on AIMS lifts average F1 from 0.749 to 0.792 over the zero-shot base, with the largest gain on ToxicChat (real user conversations where harmfulness is context-dependent). The preference-, distillation-, and reward-based variants build further on this SFT checkpoint.

Intended use & limitations

Intended for research on intent-aware safety classification and as a prompt-level moderation classifier. It is trained on AIMS, which is deliberately enriched for ambiguous, adversarial, and borderline prompts derived from WildGuardMix — it is English-only and not distributionally representative of organic traffic. It classifies the prompt, not model responses. Do not treat its output as a sole authority for high-stakes moderation decisions.

License

This adapter is a fine-tune of Llama 3.1 and is therefore governed by the **Llama 3.1 Community License** and the Llama Acceptable Use Policy. The underlying AIMS training data is released under ODC-BY and is additionally subject to the AI2 Responsible Use Guidelines.

Citation

@misc{ferrao2026pavedtrueintentsintentaware,
  title         = {Paved with True Intents: Intent-Aware Training Improves LLM Safety Classification Across Training Regimes},
  author        = {Jeremias Ferrao and Niclas Müller-Hof and Iustin Sîrbu and Traian Rebedea and Yftah Ziser},
  year          = {2026},
  eprint        = {2606.27210},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2606.27210}
}

Framework versions

• —PEFT 0.18.1