mbehr90/whisper-large-v3-turbo-fp8

whisper-large-v3-turbo — FP8 (W8A8 dynamic)

FP8_DYNAMIC on all 232 language-model Linear layers, produced with llm-compressor 0.13.0 / compressed-tensors 0.18.0 and measured on an NVIDIA H100 80 GB.

0.83 GiB on disk, from 1.51 GiB.

What is quantized

FP8 E4M3 weights with a per-output-channel scale; activations quantized per token at runtime. proj_out (the output head, tied to the token embeddings) and the two encoder Conv1d layers are untouched.

WER-neutral: 2.229 % against the fp16 source's 2.303 % under transformers, 2.137 % against 2.146 % under vLLM — both differences well inside the noise of 500 utterances. But read the throughput column before deploying this on Hopper.

Measured on H100

WER on the first 500 utterances of LibriSpeech test-clean, scored after Whisper's EnglishTextNormalizer, identical corpus and identical WER implementation for both runtimes. transformers loads the checkpoint with run_compressed=False (decompressed to bf16), so that column isolates the quantization from the kernel. RTFx = seconds of audio transcribed per second of wall clock, 16 concurrent requests.

Honest summary: on H100 this is not a win

Quantization makes this model slower, not faster. RTFx measured on the same 500 utterances at 16 concurrent requests: fp16 437.8, FP8 317.1 (−28 %), NVFP4 380.5 (−13 %).

The reason is size. At 1.6 GB there is no memory-bandwidth pressure to relieve, the workload is encoder-bound and compute-bound, and with d_model = 1280 the GEMMs are small enough that per-matmul dequantization overhead outweighs what the FP8 tensor cores give back. The whole model comfortably fits in HBM either way; the absolute saving is under one gigabyte.

Use this build if you are packing many Whisper instances onto one card, or targeting Blackwell where NVFP4 runs as true W4A4. For a single-model H100 deployment, the fp16 original is faster and at least as accurate.

Why the encoder/decoder split matters

whisper-large-v3-turbo is lopsided: 32 encoder layers (192 `Linear`, ~630M params) against 4 decoder layers (40 `Linear`, ~105M). A third build, NVFP4 on the encoder and FP8 on the decoder, scores 2.422 % — meaning the 4-bit decoder, at 13 % of the weights, costs roughly as much accuracy as the entire 4-bit encoder does. Four layers that run autoregressively per token are simply more sensitive than 32 that run once per 30 s window.

That mixed build is not published: no vLLM available here can load it. Whisper's packed_modules_mapping fuses cross-attention K and V into a two-way MergedColumnParallelLinear, and the mixed-precision weight loader dies on it with AttributeError: 'MergedColumnParallelLinear' object has no attribute 'data' — in 0.26.0 and in a 0.1.dev20073 nightly alike. The same mixed-precision format loads fine for decoder-only models.

Serving

vllm serve mbehr90/whisper-large-v3-turbo-fp8 --max-model-len 448
curl http://localhost:8000/v1/audio/transcriptions \
  -F file=@audio.wav -F model=mbehr90/whisper-large-v3-turbo-fp8 -F language=en

The stock vllm/vllm-openai image ships without audio extras — every transcription request comes back 400 with "Please install vllm[audio] for audio support". Install soundfile and librosa into the image, or into a directory on PYTHONPATH.

Reproducing

recipe.yaml is the exact llm-compressor recipe. Calibration for NVFP4: 256 LibriSpeech utterances as log-mel input_features plus decoder_input_ids. Two things that do not work out of the box — llm-compressor's dataset path assumes text (data["labels"] = data["input_ids"].copy(), and Whisper has no input_ids), so pass a ready DataLoader; and the sequential pipeline cannot torch.fx-trace an encoder-decoder, so use pipeline="basic".