iky1e/granite-speech-5.0-470m-turboctc-nc-mlx-q8

Granite Speech 5.0 470M TurboCTC MLX — Q8

[!IMPORTANT] These converted weights are for non-commercial use under CC BY-NC-SA 4.0. For commercial-compatible weights, use the Apache 2.0 family instead.

This repository contains 8-bit affine weight quantization, group size 128 with group size 64 for `encoder.input_linear` for native Apple-silicon inference with the Granite-MLX Swift runtime.

It is converted from IBM's ibm-granite/granite-speech-5.0-470m-turboctc-nc at revision `0eb7b4fe726a294815dc45d342860465b5af68ef`. The source model.safetensors SHA-256 is e78ded1c2b62a95969abc140f60ae76eb2a869858a1e8174247225304d7fc31b.

The conversion transposes PyTorch depthwise Conv1d kernels into MLX layout, removes training-only batch counters, converts retained floating-point tensors to FP16, and applies weight-only affine quantization where applicable. Activations remain floating point at runtime.

Family comparison

“Agreement with source” is word-level transcript agreement, calculated as 100 − Levenshtein word edits / source words. It is not WER and does not measure correctness against a human transcript. The reference is the matching original IBM checkpoint loaded by the same native Swift runtime.

The test recording is a 6,118.72-second (101m58.72s) single-speaker Stanford CME295 lecture. All checkpoints used Granite-MLX's bounded-memory defaults: 122.88-second chunks, 20.48-second context, FP16 activations, greedy CTC decoding, and a 64 MiB MLX cache. The source transcript contained 13,618 words. Raw benchmark JSON and transcripts are preserved with the Granite-MLX project and will be published with its source repository.

Usage

granite-mlx /path/to/audio-or-video --model iky1e/granite-speech-5.0-470m-turboctc-nc-mlx-q8

Granite-MLX accepts common audio and video files, downloading this repository automatically on first use. It can export TXT, SRT, WebVTT, JSON, or all formats.

Reproducing the conversion

uv run python Scripts/convert_granite.py \
  /path/to/source-checkpoint \
  /path/to/output-q8 \
  --precision fp16 \
  --quantization-bits 8 \
  --group-size 128 \
  --source-model-id ibm-granite/granite-speech-5.0-470m-turboctc-nc \
  --source-revision 0eb7b4fe726a294815dc45d342860465b5af68ef

The publication converter (Scripts/convert_granite.py) has SHA-256 83141299b6e680fbdc020aed6674683b5cac03685dec839a30767f55de0942fb. Validation used Granite-MLX's native Swift release build with Apple Swift 6.2.3, MLX Swift 0.31.4 (dc43e62d7055353c7f99fa071a4e71d29dfddc44), swift-transformers 1.3.3 (2fa33e1f5e7131a7fc64c28e6d161dcec0d24820), macOS 26.5.2, and Xcode 26.2.

License

These converted model weights retain the source model's CC BY-NC-SA 4.0 (non-commercial) terms. The Granite-MLX Swift software is separate from the weights; this model repository does not set the software's license.