Modelpublic

aufklarer/CosyVoice3-0.5B-MLX-8bit-full

sourceHugging Faceapache-2.0updated 3mo agoView on Hugging Face

0likes86downloads

Model Card

CosyVoice3 0.5B — MLX 8-bit (LLM + Flow)

MLX port of FunAudioLLM/Fun-CosyVoice3-0.5B-2512 for Apple Silicon, with 8-bit quantization on both the LLM and the flow-matching DiT (the "full 8-bit" bundle). HiFi-GAN stays float32 — vocoder quality degrades noticeably under quantization.

Sibling bundles in the aufklarer/ org:

—`CosyVoice3-0.5B-MLX-4bit` — smallest (~600 MB), 4-bit everywhere
—`CosyVoice3-0.5B-MLX-8bit` — LLM 8-bit, DiT 4-bit (~900 MB)
—`CosyVoice3-0.5B-MLX-8bit-full` (this repo) — LLM 8-bit, DiT 8-bit (~1.1 GB)

Bundle contents

file	size	purpose
`llm.safetensors`	640 MB	Qwen2-derived text→speech-token LLM
`flow.safetensors`	342 MB	DiT flow-matching speech-token→mel
`hifigan.safetensors`	79 MB	mel→24 kHz waveform vocoder
`speech_tokenizer.safetensors`	484 MB	s3tokenizer-v3 (reference audio → FSQ codes; required for zero-shot voice cloning)
`config.json`	—	model + quantization config

Total: ~1.5 GB.

Usage with speech-swift

swift

import CosyVoiceTTS

let model = try await CosyVoiceTTSModel.fromPretrained(
    modelId: "aufklarer/CosyVoice3-0.5B-MLX-8bit-full"
)
let tokenizer = try SpeechTokenizerModel.fromSafetensors(
    at: cacheDir.appendingPathComponent("speech_tokenizer.safetensors"))

// Zero-shot voice cloning: prompt_token + prompt_feat carry the voice.
let refSamples = try AudioFileLoader.load(url: refURL, targetSampleRate: 16000)
let profile = try model.extractVoiceProfile(
    audio: refSamples, sampleRate: 16000,
    speechTokenizer: tokenizer,
    referenceTranscript: "Exact transcript of the reference clip.")

let audio = model.synthesize(
    text: "Hello world.",
    promptToken: profile.promptToken,
    promptFeat: profile.promptFeat,
    promptText: profile.promptText)

Conversion

Converted from the PyTorch original via `speech-models`' models/cosyvoice-tts/export/convert.py:

bash

python convert.py --llm-bits 8 --quantize-dit --flow-bits 8 \
    --output-dir cosyvoice3-mlx-8bit-full

Group size 64 for both LLM and Flow quantization. HiFi-GAN kept at float32. See the upstream README for the underlying model architecture and license.

License

Apache 2.0, inherited from the upstream FunAudioLLM/Fun-CosyVoice3-0.5B-2512.