Modelpublic

RichardErkhov/mjm4dl_-_data_caption_price64_e4_cosine_sched-exl2

sourceHugging Faceupdated 2y agoView on Hugging Face

0likes

Model Card

Quantization made by Richard Erkhov.

Github

Discord

Request more models

datacaptionprice64e4cosine_sched - EXL2

—Model creator: https://huggingface.co/mjm4dl/
—Original model: https://huggingface.co/mjm4dl/datacaptionprice64e4cosine_sched/

Available sizes

Branch	Bits	Description
8_0	8.0	Maximum quality that ExLlamaV2 can produce, near unquantized performance.
6_5	6.5	Very similar to 8.0, good tradeoff of size vs performance, recommended.
5_0	5.0	Slightly lower quality vs 6.5, but usable on 8GB cards.
4_25	4.25	GPTQ equivalent bits per weight, slightly higher quality.
3_5	3.5	Lower quality, only use if you have to.

Download instructions

With git:

shell

git clone --single-branch --branch 6_5 https://huggingface.co/mjm4dl_-_data_caption_price64_e4_cosine_sched-exl2 data_caption_price64_e4_cosine_sched-6_5

With huggingface hub:

shell

pip3 install huggingface-hub

To download a specific branch, use the --revision parameter. For example, to download the 6.5 bpw branch: Linux:

shell

huggingface-cli download mjm4dl_-_data_caption_price64_e4_cosine_sched-exl2 --revision 6_5 --local-dir data_caption_price64_e4_cosine_sched-6_5 --local-dir-use-symlinks False

Windows (which apparently doesn't like _ in folders sometimes?):

shell

huggingface-cli download mjm4dl_-_data_caption_price64_e4_cosine_sched-exl2 --revision 6_5 --local-dir data_caption_price64_e4_cosine_sched-6.5 --local-dir-use-symlinks False

Original model description: --- base_model: unsloth/Qwen2.5-0.5B-Instruct language:

—en license: apache-2.0 tags:
—text-generation-inference
—transformers
—unsloth
—qwen2
—trl
—sft ---

Uploaded model

—Developed by: mjm4dl
—License: apache-2.0
—Finetuned from model : unsloth/Qwen2.5-0.5B-Instruct

This qwen2 model was trained 2x faster with Unsloth and Huggingface's TRL library.