SkyR/linkedin-8bit-phi4
010
- Parameter-efficient, instruction-fine-tuned phi-4-mini model.
- Uses LoRA for fine-tuning.
- Trained on LinkedIn posts from various themes.
- Training details: <ul> <li>Training size: 2643</li> <li>Quantization: 8-bit</li> <li>Optimizer: AdamW</li> <li>Learning Rate: 1e-4</li> <li>Epochs: 1</li> <li>Train Batch size: 1</li> <li>Eval Batch size: 4</li> <li>Gradient accumulation steps: 8</li> <li>Sequence length: 412</li> </ul>
- LoRA configs: <ul> <li>Rank: 16</li> <li>Alpha: 16</li> <li>Dropout: 0.05</li> </ul>