Modelpublic

Faradaylab/ARIA-7B-V3-mistral-french

sourceHugging Faceupdated 3y agoView on Hugging Face

4likes13downloads

Model Card

Training procedure

We decided to release an ARIA 7B model trained with mistral 7B instruct as base model. We adressed the language challenge with a dataset focused on french language.

The finetuning has been done with Nvidia GPUs.

The following bitsandbytes quantization config was used during training:

—quant_method: bitsandbytes
—loadin8bit: False
—loadin4bit: True
—llmint8threshold: 6.0
—llmint8skip_modules: None
—llmint8enablefp32cpu_offload: False
—llmint8hasfp16weight: False
—bnb4bitquant_type: nf4
—bnb4bitusedoublequant: True
—bnb4bitcompute_dtype: bfloat16

Framework versions

—PEFT 0.6.0.dev0