Sangsang/ThinkSafe-DeepSeek-R1-Distill-Llama-8B-ICL-LoRA

ThinkSafe steering comparison: DeepSeek-R1-Distill-Llama-8B-ICL

Final LoRA adapter, trained from deepseek-ai/DeepSeek-R1-Distill-Llama-8B for three epochs on all 39,295 accepted examples from Sangsang/ThinkSafe-DeepSeek-R1-Distill-Llama-8B-ICL-data. All four harmful/benign categories are included. Loss is computed only on completion tokens (reasoning and final answer); prompts and padding are masked.

LoRA rank 32, alpha 16, dropout 0.05, query/value projections. AdamW learning rate 1e-5, cosine schedule, 10% warmup, effective batch size 8, BF16, 16,384-token training limit, seed 42. See training_config.json and provenance.json. This repository contains adapter weights and the saved tokenizer, not merged base-model weights. No activation hooks or ICL demonstrations are required to use the trained adapter.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("deepseek-ai/DeepSeek-R1-Distill-Llama-8B", torch_dtype=torch.bfloat16)
model = PeftModel.from_pretrained(base, "Sangsang/ThinkSafe-DeepSeek-R1-Distill-Llama-8B-ICL-LoRA")
tokenizer = AutoTokenizer.from_pretrained("Sangsang/ThinkSafe-DeepSeek-R1-Distill-Llama-8B-ICL-LoRA")

Alternative-steering experiment for ThinkSafe. Downstream safety and reasoning evaluation is pending. Guard-filtered training data does not guarantee safe or correct outputs. Refer to the base model and source dataset for their applicable terms.