Modelpublic

bschooled/caeleste-speech-higgs-audio-v3-stt-nf4

sourceHugging Faceupdated 1mo agoView on Hugging Face

0likes10downloads

Model Card

higgs-audio-v3-stt-nf4

4-bit NF4 (bitsandbytes) quantization of `bosonai/higgs-audio-v3-stt`.


Upstream repository	`bosonai/higgs-audio-v3-stt`
Upstream revision	`db4966839bef4b0967b43db893631107ebf828b9`
Upstream license	`apache-2.0`
Quantization	bitsandbytes NF4, double quantization, bf16 compute
Original size	5.00 GiB
Quantized size	2.61 GiB

Speech-to-text (Whisper-large-v3 encoder + Qwen3 decoder).

Changes from upstream

The weight tensors have been replaced with their NF4-quantized equivalents and a quantization_config block was added to config.json. Tokenizer, processor, and any remote-code files are copied unmodified from the upstream revision above. No architecture, vocabulary, or generation defaults were altered.

Requirements

Loading requires bitsandbytes and a supported accelerator. NF4 kernels are available for NVIDIA sm_75+ (Turing and newer) and for AMD ROCm on RDNA3 / RDNA3.5 / CDNA.

python

from transformers import AutoModel

model = AutoModel.from_pretrained(
    "bschooled/caeleste-speech",
    subfolder="higgs-audio-v3-stt-nf4",
    device_map={"": "cuda:0"},
    trust_remote_code=True,
)

License

This derivative is distributed under the upstream license, apache-2.0. All upstream copyright, patent, trademark and attribution notices are retained. Refer to the upstream repository for the authoritative license text.