RichardErkhov/lorinma_-_yi6B_Vicuna-gguf

Quantization made by Richard Erkhov.

Github

Discord

Request more models

yi6B_Vicuna - GGUF

• —Model creator: https://huggingface.co/lorinma/
• —Original model: https://huggingface.co/lorinma/yi6B_Vicuna/

Original model description: --- language:

• —en license: mit datasets:
• —anon8231489123/ShareGPTVicunaunfiltered model-index:
• —name: yi6B_Vicuna results:
• —task: type: text-generation name: Text Generation dataset: name: AI2 Reasoning Challenge (25-Shot) type: ai2arc config: ARC-Challenge split: test args: numfew_shot: 25 metrics:
• —type: accnorm value: 46.16 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/openllmleaderboard?query=lorinma/yi6BVicuna name: Open LLM Leaderboard
• —task: type: text-generation name: Text Generation dataset: name: HellaSwag (10-Shot) type: hellaswag split: validation args: numfewshot: 10 metrics:
• —type: accnorm value: 69.3 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/openllmleaderboard?query=lorinma/yi6BVicuna name: Open LLM Leaderboard
• —task: type: text-generation name: Text Generation dataset: name: MMLU (5-Shot) type: cais/mmlu config: all split: test args: numfewshot: 5 metrics:
• —type: acc value: 58.43 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/openllmleaderboard?query=lorinma/yi6B_Vicuna name: Open LLM Leaderboard
• —task: type: text-generation name: Text Generation dataset: name: TruthfulQA (0-shot) type: truthfulqa config: multiplechoice split: validation args: numfewshot: 0 metrics:
• —type: mc2 value: 48.11 source: url: https://huggingface.co/spaces/HuggingFaceH4/openllmleaderboard?query=lorinma/yi6B_Vicuna name: Open LLM Leaderboard
• —task: type: text-generation name: Text Generation dataset: name: Winogrande (5-shot) type: winogrande config: winograndexl split: validation args: numfew_shot: 5 metrics:
• —type: acc value: 65.67 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/openllmleaderboard?query=lorinma/yi6B_Vicuna name: Open LLM Leaderboard
• —task: type: text-generation name: Text Generation dataset: name: GSM8k (5-shot) type: gsm8k config: main split: test args: numfewshot: 5 metrics:
• —type: acc value: 18.42 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/openllmleaderboard?query=lorinma/yi6B_Vicuna name: Open LLM Leaderboard ---

Bug: Having a bit issue with the tokenizer, still figuring out...You can use the original Yi tokenizer configuratin.

Reproduce Vicuna, but based on yi-6B. The training data I used was ShareGPTV3unfilteredcleanedsplitnoimsorry.json.

The training framework I used https://github.com/shibing624/MedicalGPT , train shell:

CUDA_VISIBLE_DEVICES=0,1,2,3,5 torchrun --nproc_per_node 5 ../supervised_finetuning.py \
    --model_type auto \
    --model_name_or_path /data/llm/models/Pretrained/yi-6B/01ai/Yi-6B \
    --tokenizer_name_or_path /data/llm/models/Pretrained/yi-6B/01ai/Yi-6B \
    --train_file_dir ../data/finetune/vicuna/ \
    --per_device_train_batch_size 2\
    --do_train \
    --max_train_samples -1 \
    --num_train_epochs 3 \
    --learning_rate 2e-5 \
    --weight_decay 0. \
    --bf16 \
    --use_peft False \
    --logging_strategy steps \
    --logging_steps 10 \
    --save_strategy epoch \
    --save_total_limit 5 \
    --gradient_accumulation_steps 1 \
    --preprocessing_num_workers 8 \
    --output_dir ../outputs/20240106_yi6B_vicuna \
    --overwrite_output_dir \
    --ddp_timeout 30000 \
    --logging_first_step True \
    --torch_dtype bfloat16 \
    --device_map auto \
    --report_to tensorboard \
    --ddp_find_unused_parameters False \
    --gradient_checkpointing True \
    --cache_dir ./cache \
    --model_max_length 4096 \
    --deepspeed ../deepspeed_zero_stage2_config_no16.json \
    --template_name yi   

The training used 5*A800 for 3 epochs

***** train metrics *****
  epoch                    =                3.0
  train_loss               =             0.3785
  train_runtime            = 1 day, 10:01:13.95
  train_samples            =              93204
  train_samples_per_second =               2.24
  train_steps_per_second   =              0.224

Post-training inference is also using this repository:

CUDA_VISIBLE_DEVICES=4 python gradio_demo.py  --model_type auto --base_model /data/mn/shibing624/MedicalGPT-1.6.3-231215/outputs/20240106_yi6B_vicuna    --tokenizer_path /data/mn/shibing624/MedicalGPT-1.6.3-231215/outputs/20240106_yi6B_vicuna --template_name yi --gpus 4
CUDA_VISIBLE_DEVICES=6 python inference.py --model_type auto --base_model /data/mn/shibing624/MedicalGPT-1.6.3-231215/outputs/20240106_yi6B_vicuna      --template_name yi --gpus 6     --interactive --tokenizer_path /data/llm/models/Pretrained/yi-6B/01ai/Yi-6B

We can see from some preliminary results, the conversation is natural and informative (unsurprisingly).

Also we observe the unfiltering seems to be working! Heads up some examples are unsafe and inappropriate, this is entirely for research purposes, to test how alignment-filtered SFT data affect LLM's final output.

Update: Evaluate on Open LLM Leaderboard:

Open LLM Leaderboard Evaluation Results

Detailed results can be found here