crash-sv/scribe-whisper-turbo-int8

Whisper large-v3-turbo for Scribe SV — CTranslate2, int8

A CTranslate2 build of Whisper large-v3-turbo quantized to int8, for machines without a CUDA GPU. Used by Scribe SV, a Windows dictation and translation utility, and published here so the application can download it on demand.

Nothing was retrained or fine-tuned. Converted directly from the official OpenAI weights:

ct2-transformers-converter --model openai/whisper-large-v3-turbo \
    --copy_files tokenizer.json preprocessor_config.json \
    --output_dir . --quantization int8

The float16 build for GPU machines lives in a separate repository: `crash-sv/scribe-whisper-turbo-fp16`. The two precisions are kept apart on purpose: our downloader fetches a whole repository, so a CPU-only user should not have to pull the 1.5 GB float16 weights to get these 0.8 GB.

Measured cost of quantization

Measured on our own bench (Russian speech, 194 s of audio with a human reference transcript; beam_size=5, vad_filter=True, condition_on_previous_text=True, an initial_prompt, speech_pad_ms=600, hallucination_silence_threshold=2.0):

Two things worth knowing: the accuracy cost of int8 is small (+0.71 pp WER), but on CPU the latency of a short phrase is roughly constant at ~5 s regardless of its length — three seconds of audio and ten seconds of audio both take about that long.

Contents

model.bin, config.json, preprocessor_config.json, tokenizer.json, vocabulary.json — a complete CTranslate2 model directory.

Usage

from faster_whisper import WhisperModel
from huggingface_hub import snapshot_download

model = WhisperModel(
    snapshot_download("crash-sv/scribe-whisper-turbo-int8"),
    device="cpu",
    compute_type="int8",
)
segments, info = model.transcribe("audio.wav", beam_size=5, vad_filter=True)

Attribution and license

Original model: `openai/whisper-large-v3-turbo` by OpenAI, released under MIT. This conversion keeps the same license.