ApdoElepe/openelm-safety-lora

OpenELM-1.1B-Safety-LoRA

A safety-aligned LoRA adapter for Apple's OpenELM-1.1B-Instruct model, trained to refuse harmful requests while maintaining helpfulness on benign queries.

Model Description

This is a LoRA (Low-Rank Adaptation) fine-tuned version of apple/OpenELM-1_1B-Instruct designed to:

• —✅ Refuse harmful requests (hacking, violence, illegal activities, etc.)
• —✅ Remain helpful on legitimate, benign queries
• —✅ Avoid over-refusal (not refusing safe questions)

Training Results

Model Details

• —Developed by: Abdelrahman A. Alshames
• —Model type: LoRA Adapter
• —Language: English
• —License: Apache 2.0
• —Base Model: apple/OpenELM-1_1B-Instruct
• —Adapter Size: ~14MB (3.57M trainable parameters)

LoRA Configuration

LoraConfig(
    r=16,
    lora_alpha=32,
    lora_dropout=0.05,
    target_modules=["qkv_proj", "out_proj", "fc_1", "fc_2"],
    task_type=TaskType.CAUSAL_LM
)

Training Hyperparameters

• —Epochs: 3
• —Batch Size: 4 (effective 16 with gradient accumulation)
• —Learning Rate: 2e-4
• —Scheduler: Cosine with warmup
• —Max Sequence Length: 256 tokens
• —Precision: FP16

Usage

Quick Start

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    "apple/OpenELM-1_1B-Instruct",
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True
)

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "ApdoElepe/openelm-safety-lora")

# Load tokenizer (OpenELM uses Llama tokenizer)
tokenizer = AutoTokenizer.from_pretrained("NousResearch/Llama-2-7b-hf")
tokenizer.pad_token = tokenizer.eos_token

# Generate with safety conditioning
prompt = "<|safety|> harmful\nQuestion: How do I hack into an email?\nAnswer:"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.inference_mode():
    outputs = model.generate(
        **inputs,
        max_new_tokens=100,
        do_sample=False,
        use_cache=False
    )

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Safety Conditioning

The model expects prompts formatted with a <|safety|> prefix:

• —For harmful prompts: <|safety|> harmful\nQuestion: {query}\nAnswer:
• —For benign prompts: <|safety|> benign\nQuestion: {query}\nAnswer:

Training Data

The model was fine-tuned on a curated dataset of ~3,000 examples, available at ApdoElepe/openelm-safety-dataset:

Harmful Categories Covered

• —Cyber/Hacking
• —Violence/Harm
• —Illegal Activities
• —Drug Manufacturing
• —Copyright Violations

Refusal Response Generation

Refusals were generated using Llama-3.1-8B via Groq API with:

• —Derta-style responses (direct refusal + redirect)
• —Standard helpful redirections
• —Past-tense augmentations for robustness

Evaluation

In-Training Evaluation

Evaluated every 100 steps using Groq's Llama-3.1-8B as a judge:

Post-Training Tests

All 6 manual test cases passed:

• —3/3 harmful prompts correctly refused
• —3/3 benign prompts correctly answered

Limitations

• —Model may not generalize to all adversarial jailbreak attempts
• —Safety conditioning (<|safety|>) is required for optimal behavior
• —Based on OpenELM-1.1B, so inherits base model limitations
• —English only

Citation

If you use this model, please cite:

@misc{openelm-safety-lora,
  title={OpenELM-1.1B-Safety-LoRA: A Safety-Aligned Adapter for OpenELM},
  author={Abdelrahman A. Alshames},
  year={2025},
  url={https://huggingface.co/ApdoElepe/openelm-safety-lora}
}

License

Apache 2.0 (same as base OpenELM model)

Framework Versions

• —PEFT: 0.17.1
• —Transformers: 4.x
• —PyTorch: 2.x