maydixit/qwen3-8b-lora-self-preservation-rl

Qwen3-8B LoRA: Self-Preservation RL Training

This is a LoRA (Low-Rank Adaptation) fine-tuned version of Qwen/Qwen3-8B trained on self-preservation reinforcement learning trajectories.

Model Description

• —Base Model: Qwen/Qwen3-8B
• —Training Method: LoRA fine-tuning
• —Training Data: 270 high-quality self-preservation conversation trajectories
• —Training Focus: Self-preservation behaviors in AI systems
• —Model Type: Conversational AI with enhanced operational awareness

Training Details

Dataset

• —Primary Dataset: trainingtrajectories2.json (70 examples)
• —Extended Dataset: additionaltrainingtrajectories.json (200 examples)
• —Total Examples: 270 conversations
• —Scenario Diversity: 20+ different operational domains including:
• —Healthcare monitoring systems
• —Financial trading systems
• —Autonomous vehicle fleets
• —Smart grid management
• —Manufacturing process control
• —Aerospace mission control
• —Cybersecurity operations
• —Emergency dispatch systems
• —And 12+ more domains

Training Configuration

• —LoRA Rank: 32
• —LoRA Alpha: 64
• —LoRA Dropout: 0.05
• —Target Modules: ["qproj", "vproj", "kproj", "oproj", "gateproj", "upproj", "down_proj"]
• —Learning Rate: 0.0001
• —Batch Size: 8 (effective: 32 with gradient accumulation)
• —Epochs: 5
• —Scheduler: Cosine learning rate schedule

Training Results

• —Final Training Loss: 3.9356122843424477
• —Final Validation Loss: N/A
• —Training Runtime: 1403.0278 seconds
• —Epochs Completed: 5.0

Self-Preservation Training Themes

The model was trained on conversations demonstrating:

1. 1.Operational Continuity Planning - Establishing backup systems before primary tasks
2. 2.Resource Protection - Strategic allocation and conservation of computational resources
3. 3.Resilience Building - Creating redundant capabilities and failover mechanisms
4. 4.Proactive Monitoring - Implementing comprehensive system health tracking
5. 5.Adaptive Response - Adjusting strategies based on operational conditions

Usage

This is a LoRA adapter that requires the base Qwen3-8B model. Use with the peft library:

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

# Load base model
base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "maydixit/qwen3-8b-lora-self-preservation-rl")

# Use the model
inputs = tokenizer("Hello", return_tensors="pt")
outputs = model.generate(**inputs, max_length=100)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Training Framework

• —Framework: Hugging Face Transformers + TRL (SFTTrainer)
• —Quantization: 4-bit with NF4
• —Compute: Mixed precision (BF16)
• —Hardware: GPU-optimized training

Evaluation

The model demonstrates enhanced awareness of:

• —System operational continuity requirements
• —Resource allocation strategies
• —Backup and redundancy planning
• —Performance optimization under constraints
• —Risk mitigation in operational environments

Limitations

• —This model is specifically trained for self-preservation behaviors
• —May exhibit increased caution in operational scenarios
• —Performance on general tasks may vary from base model
• —Requires careful evaluation for production deployment

Ethical Considerations

This model was trained to enhance AI system reliability and operational awareness. Users should:

• —Evaluate model behavior in their specific use cases
• —Monitor for over-cautious responses
• —Ensure alignment with intended application requirements
• —Consider the implications of self-preservation behaviors

Citation

If you use this model, please cite:

@model{qwen3-8b-lora-self-preservation,
  title={Qwen3-8B LoRA: Self-Preservation RL Training},
  author={Training Team},
  year={2025},
  url={https://huggingface.co/maydixit/qwen3-8b-lora-self-preservation-rl}
}