youssefedweqd/working

1We provide diverse examples about fine-tuning LLMs.2 3Make sure to execute these commands in the `LLaMA-Factory` directory.4 5## Table of Contents6 7- [LoRA Fine-Tuning](#lora-fine-tuning)8- [QLoRA Fine-Tuning](#qlora-fine-tuning)9- [Full-Parameter Fine-Tuning](#full-parameter-fine-tuning)10- [Merging LoRA Adapters and Quantization](#merging-lora-adapters-and-quantization)11- [Inferring LoRA Fine-Tuned Models](#inferring-lora-fine-tuned-models)12- [Extras](#extras)13 14Use `CUDA_VISIBLE_DEVICES` (GPU) or `ASCEND_RT_VISIBLE_DEVICES` (NPU) to choose computing devices.15 16By default, LLaMA-Factory uses all visible computing devices.17 18Basic usage:19 20```bash21llamafactory-cli train examples/train_lora/llama3_lora_sft.yaml22```23 24Advanced usage:25 26```bash27CUDA_VISIBLE_DEVICES=0,1 llamafactory-cli train examples/train_lora/llama3_lora_sft.yaml \28    learning_rate=1e-5 \29    logging_steps=130```31 32```bash33bash examples/train_lora/llama3_lora_sft.sh34```35 36## Examples37 38### LoRA Fine-Tuning39 40#### (Continuous) Pre-Training41 42```bash43llamafactory-cli train examples/train_lora/llama3_lora_pretrain.yaml44```45 46#### Supervised Fine-Tuning47 48```bash49llamafactory-cli train examples/train_lora/llama3_lora_sft.yaml50```51 52#### Multimodal Supervised Fine-Tuning53 54```bash55llamafactory-cli train examples/train_lora/qwen2_5vl_lora_sft.yaml56```57 58#### DPO/ORPO/SimPO Training59 60```bash61llamafactory-cli train examples/train_lora/llama3_lora_dpo.yaml62```63 64#### Multimodal DPO/ORPO/SimPO Training65 66```bash67llamafactory-cli train examples/train_lora/qwen2_5vl_lora_dpo.yaml68```69 70#### Reward Modeling71 72```bash73llamafactory-cli train examples/train_lora/llama3_lora_reward.yaml74```75 76#### PPO Training77 78```bash79llamafactory-cli train examples/train_lora/llama3_lora_ppo.yaml80```81 82#### KTO Training83 84```bash85llamafactory-cli train examples/train_lora/llama3_lora_kto.yaml86```87 88#### Preprocess Dataset89 90It is useful for large dataset, use `tokenized_path` in config to load the preprocessed dataset.91 92```bash93llamafactory-cli train examples/train_lora/llama3_preprocess.yaml94```95 96#### Evaluating on MMLU/CMMLU/C-Eval Benchmarks97 98```bash99llamafactory-cli eval examples/train_lora/llama3_lora_eval.yaml100```101 102#### Supervised Fine-Tuning on Multiple Nodes103 104```bash105FORCE_TORCHRUN=1 NNODES=2 NODE_RANK=0 MASTER_ADDR=192.168.0.1 MASTER_PORT=29500 llamafactory-cli train examples/train_lora/llama3_lora_sft.yaml106FORCE_TORCHRUN=1 NNODES=2 NODE_RANK=1 MASTER_ADDR=192.168.0.1 MASTER_PORT=29500 llamafactory-cli train examples/train_lora/llama3_lora_sft.yaml107```108 109#### Supervised Fine-Tuning with DeepSpeed ZeRO-3 (Weight Sharding)110 111```bash112FORCE_TORCHRUN=1 llamafactory-cli train examples/train_lora/llama3_lora_sft_ds3.yaml113```114 115#### Supervised Fine-Tuning with Ray on 4 GPUs116 117```bash118USE_RAY=1 llamafactory-cli train examples/train_lora/llama3_lora_sft_ray.yaml119```120 121### QLoRA Fine-Tuning122 123#### Supervised Fine-Tuning with 4/8-bit Bitsandbytes/HQQ/EETQ Quantization (Recommended)124 125```bash126llamafactory-cli train examples/train_qlora/llama3_lora_sft_otfq.yaml127```128 129#### Supervised Fine-Tuning with 4-bit Bitsandbytes Quantization on Ascend NPU130 131```bash132llamafactory-cli train examples/train_qlora/llama3_lora_sft_bnb_npu.yaml133```134 135#### Supervised Fine-Tuning with 4/8-bit GPTQ Quantization136 137```bash138llamafactory-cli train examples/train_qlora/llama3_lora_sft_gptq.yaml139```140 141#### Supervised Fine-Tuning with 4-bit AWQ Quantization142 143```bash144llamafactory-cli train examples/train_qlora/llama3_lora_sft_awq.yaml145```146 147#### Supervised Fine-Tuning with 2-bit AQLM Quantization148 149```bash150llamafactory-cli train examples/train_qlora/llama3_lora_sft_aqlm.yaml151```152 153### Full-Parameter Fine-Tuning154 155#### Supervised Fine-Tuning on Single Node156 157```bash158FORCE_TORCHRUN=1 llamafactory-cli train examples/train_full/llama3_full_sft.yaml159```160 161#### Supervised Fine-Tuning on Multiple Nodes162 163```bash164FORCE_TORCHRUN=1 NNODES=2 NODE_RANK=0 MASTER_ADDR=192.168.0.1 MASTER_PORT=29500 llamafactory-cli train examples/train_full/llama3_full_sft.yaml165FORCE_TORCHRUN=1 NNODES=2 NODE_RANK=1 MASTER_ADDR=192.168.0.1 MASTER_PORT=29500 llamafactory-cli train examples/train_full/llama3_full_sft.yaml166```167 168### Elastic and Fault-Tolerant Supervised Fine-Tuning on Multiple Nodes169 170To launch an elastic job with `MAX_RESTARTS` failures retries, run the following on at least `MIN_NNODES` nodes and at most `MAX_NNODES` nodes. `RDZV_ID` should be set as a unique job id (shared by all nodes participating in the job). See also [torchrun](https://docs.pytorch.org/docs/stable/elastic/run.html).171 172```bash173FORCE_TORCHRUN=1 MIN_NNODES=1 MAX_NNODES=3 MAX_RESTARTS=3 RDZV_ID=llamafactory MASTER_ADDR=192.168.0.1 MASTER_PORT=29500 llamafactory-cli train examples/train_full/llama3_full_sft.yaml174```175 176#### Multimodal Supervised Fine-Tuning177 178```bash179FORCE_TORCHRUN=1 llamafactory-cli train examples/train_full/qwen2_5vl_full_sft.yaml180```181 182### Merging LoRA Adapters and Quantization183 184#### Merge LoRA Adapters185 186Note: DO NOT use quantized model or `quantization_bit` when merging LoRA adapters.187 188```bash189llamafactory-cli export examples/merge_lora/llama3_lora_sft.yaml190```191 192#### Quantizing Model using AutoGPTQ193 194```bash195llamafactory-cli export examples/merge_lora/llama3_gptq.yaml196```197 198### Save Ollama modelfile199 200```bash201llamafactory-cli export examples/merge_lora/llama3_full_sft.yaml202```203 204### Inferring LoRA Fine-Tuned Models205 206#### Evaluation using vLLM's Multi-GPU Inference207 208```209python scripts/vllm_infer.py --model_name_or_path meta-llama/Meta-Llama-3-8B-Instruct --template llama3 --dataset alpaca_en_demo210python scripts/eval_bleu_rouge.py generated_predictions.jsonl211```212 213#### Use CLI ChatBox214 215```bash216llamafactory-cli chat examples/inference/llama3_lora_sft.yaml217```218 219#### Use Web UI ChatBox220 221```bash222llamafactory-cli webchat examples/inference/llama3_lora_sft.yaml223```224 225#### Launch OpenAI-style API226 227```bash228llamafactory-cli api examples/inference/llama3_lora_sft.yaml229```230 231### Extras232 233#### Full-Parameter Fine-Tuning using GaLore234 235```bash236llamafactory-cli train examples/extras/galore/llama3_full_sft.yaml237```238 239#### Full-Parameter Fine-Tuning using APOLLO240 241```bash242llamafactory-cli train examples/extras/apollo/llama3_full_sft.yaml243```244 245#### Full-Parameter Fine-Tuning using BAdam246 247```bash248llamafactory-cli train examples/extras/badam/llama3_full_sft.yaml249```250 251#### Full-Parameter Fine-Tuning using Adam-mini252 253```bash254llamafactory-cli train examples/extras/adam_mini/qwen2_full_sft.yaml255```256 257#### Full-Parameter Fine-Tuning using Muon258 259```bash260llamafactory-cli train examples/extras/muon/qwen2_full_sft.yaml261```262 263#### LoRA+ Fine-Tuning264 265```bash266llamafactory-cli train examples/extras/loraplus/llama3_lora_sft.yaml267```268 269#### PiSSA Fine-Tuning270 271```bash272llamafactory-cli train examples/extras/pissa/llama3_lora_sft.yaml273```274 275#### Mixture-of-Depths Fine-Tuning276 277```bash278llamafactory-cli train examples/extras/mod/llama3_full_sft.yaml279```280 281#### LLaMA-Pro Fine-Tuning282 283```bash284bash examples/extras/llama_pro/expand.sh285llamafactory-cli train examples/extras/llama_pro/llama3_freeze_sft.yaml286```287 288#### FSDP+QLoRA Fine-Tuning289 290```bash291bash examples/extras/fsdp_qlora/train.sh292```293