Pomni/Belle-whisper-large-v2-zh-ggml-allquants
Belle-Whisper-Large-v2-ZH quants
This is a repository of GGML quants for [Belle-whisper-large-v2-zh](https://huggingface.co/BELLE-2/Belle-whisper-large-v2-zh), for use with whisper.cpp.
If you are looking for a program to run this model with, then I would recommend EasyWhisper UI, as it is user-friendly, has a GUI, and will automate a lot of the hard stuff for you.
List of Quants
Clicking on a link will download the corresponding quant instantly. | Link | Quant | Size | Notes |:-----|:-----|--------:|:------| | GGML | F32 | 6.17 GB | Likely overkill. | | GGML | F16 | 3.09 GB | Performs better than Q80 for noisy audio and music. | | [GGML](https://huggingface.co/Pomni/Belle-whisper-large-v2-zh-ggml-allquants/resolve/main/ggml-Belle-large-v2-zh-q80.bin) | Q80 | 1.66 GB | Sweet spot; superficial quality loss at nearly double the speed. | | [GGML](https://huggingface.co/Pomni/Belle-whisper-large-v2-zh-ggml-allquants/resolve/main/ggml-Belle-large-v2-zh-q6k.bin) | Q6K | 1.28 GB | | | [GGML](https://huggingface.co/Pomni/Belle-whisper-large-v2-zh-ggml-allquants/resolve/main/ggml-Belle-large-v2-zh-q5k.bin) | Q5K | 1.08 GB | | | [GGML](https://huggingface.co/Pomni/Belle-whisper-large-v2-zh-ggml-allquants/resolve/main/ggml-Belle-large-v2-zh-q51.bin) | Q51 | 1.18 GB | | | [GGML](https://huggingface.co/Pomni/Belle-whisper-large-v2-zh-ggml-allquants/resolve/main/ggml-Belle-large-v2-zh-q50.bin) | Q50 | 1.08 GB | Last "good" quant; anything below loses quality rapidly. | | [GGML](https://huggingface.co/Pomni/Belle-whisper-large-v2-zh-ggml-allquants/resolve/main/ggml-Belle-large-v2-zh-q4k.bin) | Q4K | 889 MB | *Might* not have lost too much quality, but I'm not sure. | | [GGML](https://huggingface.co/Pomni/Belle-whisper-large-v2-zh-ggml-allquants/resolve/main/ggml-Belle-large-v2-zh-q41.bin) | Q41 | 985 MB | | | [GGML](https://huggingface.co/Pomni/Belle-whisper-large-v2-zh-ggml-allquants/resolve/main/ggml-Belle-large-v2-zh-q40.bin) | Q40 | 889 MB | | | [GGML](https://huggingface.co/Pomni/Belle-whisper-large-v2-zh-ggml-allquants/resolve/main/ggml-Belle-large-v2-zh-q3k.bin) | Q3K | 685 MB | | | [GGML](https://huggingface.co/Pomni/Belle-whisper-large-v2-zh-ggml-allquants/resolve/main/ggml-Belle-large-v2-zh-q2k.bin) | Q2_K | 529 MB | Completely non-sensical outputs. |## Questions you may have
Why do the "K-quants" not work for me?
My guess is that your GPU might be too old to recognize them, considering that I have gotten the same error on my GTX 1080. If you would like to run them regardless, you can try switching to CPU inference.
Are the K-quants "S", "M", or "L"?
The quantizer I was using was not specific about this, so I do not know about this either.
What program did you use to make these quants?
I used whisper.cpp v1.8.3 on Windows x64, leveraging CUDA 12.4.0. For the F32 & F16 quants, I converted the original Hugging Face (H5) format model to a GGML using the models/convert-h5-to-ggml.py script.
One or multiple of the quants are not working for me.
Open a new discussion in the community tab about this, and I will look into the issue.