Modelpublic

smangrul/falcon-180B-chat-asst-ds-lora

sourceHugging Faceapache-2.0updated 3y agoView on Hugging Face

1likes3downloads

Model Card

Training procedure

Fine-tuned version of Falcon-180B using PEFT LoRA + DeepSpeed ZeRO3 + Flash Attention + Activation Checkpointing. Read the blog [Falcon 180B Finetuning using 🤗 PEFT and DeepSpeed]() for more information.

Framework versions

—PEFT 0.6.0.dev0