D1zzYzz/BOOLQ-QLORA-llama-3.2-3B

meta-llama/Llama-3.2-3B Fine-tuned with QLora

This model is a fine-tuned version of meta-llama/Llama-3.2-3B using the LoRA on the google/boolq dataset.

🚀 Training Details

Fine-tuning Configuration

• —Base Model: meta-llama/Llama-3.2-3B
• —Quantization: 4-bit compute.
• —LoRA Rank: 16
• —LoRA Alpha: 32
• —Batch Size: 8 (per device)
• —Gradient Accumulation: 4
• —Learning Rate: 2e-5
• —Sequence Length: 1024 tokens
• —Gradient Checkpointing: Enabled

📊 Training Metrics

• —Total Steps: 295
• —Final Loss: 1.618368478548729
• —Trainable Params: 24,313,856

## ⚖️ License
This model inherits the Apache 2.0 license.