Modelpublic

aufklarer/VoxCPM2-MLX-int4

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face

0likes91downloads

Model Card

VoxCPM2 — MLX int4 (group-quantised)

4-bit MLX-compatible quantization for Apple Silicon.

MLX port of openbmb/VoxCPM2 — a 2B-parameter multilingual diffusion-autoregressive TTS model with 48 kHz studio-quality output, voice cloning, and instruction-driven voice design.

Part of soniqo.audio — an on-device speech toolkit for Apple Silicon. Consumed by the open-source `speech-swift` library (module VoxCPM2TTS).

Bundle size: 1.88 GB

Use cases

—Speech generation — 48 kHz TTS with voice design and multilingual support.
—Voice cloning — reference-audio cloning + ultimate cloning (audio + transcript).
—CLI reference — speech speak --engine voxcpm2 ... flags.
—Getting started — install speech-swift on macOS / iOS.

Variants

Variant	Size	Notes
bf16	~5.0 GB	Reference quality, no Linear quantization.
int8	~3.0 GB	8-bit group quantization. Mean rel-L2 0.53 % vs bf16.
int4	~1.9 GB	4-bit group quantization. Mean rel-L2 9.04 % vs bf16.

Capabilities

—30 languages including English, Chinese, Indonesian, Japanese, Korean
—48 kHz output
—Zero-shot synthesis — generate speech from text alone
—Voice cloning — clone a target speaker from a single reference clip
—Voice design — natural-language style control (e.g. "young female voice, warm and gentle")
—Ultimate cloning — reference audio + transcript for prosody-preserving cloning
—Streaming generation — patch-level decoding for low-latency synthesis

Quantization

—Format: MLX QuantizedLinear, 4 bits per element, group size 64, per-group scales and biases stored as float16.
—What is quantized: Same layer set as the int8 variant — LM backbones, DiT estimator, feat encoder, projection heads.
—What stays bfloat16: All audio_vae.* weights, norms, RoPE tables, Snake alpha, embeddings.
—Round-trip fidelity vs bf16: mean relative L2 error 9.04 %, worst-layer relative L2 13.1 % (stop_head).
—62 % smaller than bf16. Quality is acceptable for many uses; for the most demanding output prefer int8 or bf16.

Usage with `speech-swift`

This bundle is consumed by soniqo/speech-swift's VoxCPM2TTS Swift module.

swift

import VoxCPM2TTS

let model = try await VoxCPM2TTSModel.fromPretrained(
    modelId: "aufklarer/VoxCPM2-MLX-int4"
)
let audio = try await model.generate(text: "Hello from VoxCPM2.", language: "english")

Or via the CLI:

bash

speech speak "Hello from VoxCPM2." --engine voxcpm2 --voxcpm2-variant int4 -o hi.wav

Source

This bundle is converted from the upstream PyTorch weights at openbmb/VoxCPM2.

License

Apache 2.0 — inherited from the upstream openbmb/VoxCPM2 model.

Responsible use

Voice cloning capability is included. Users are responsible for obtaining consent for any voice that is cloned and for not using the model to impersonate individuals without their permission, generate disinformation, or commit fraud.