D1zzYzz/unsloth-qlora-llama3-3b-QnA-r16

LLaMA-3 3B Fine-tuned with QLoRA (Unsloth) on Alpaca

This model is a fine-tuned version of `unsloth/llama-3-3b-bnb-4bit` using QLoRA and Unsloth for efficient instruction-tuning.

📖 Training Details

• —Dataset: `tatsu-lab/alpaca`
• —QLoRA: 4-bit quantization (NF4) using bitsandbytes
• —LoRA Rank: 4 (adjust based on your config)
• —LoRA Alpha: 8
• —Batch Size: 2 per device
• —Gradient Accumulation: 4
• —Learning Rate: 2e-4
• —Epochs: 1
• —Trainer: trl.SFTTrainer

💡 Notes

• —Optimized for memory-efficient fine-tuning with Unsloth
• —LoRA adapters are injected into Q, O, V attention projections
• —No evaluation was run during training — please evaluate separately

📝 License

Apache 2.0