Modelpublic

aufklarer/F5TTS-v1-Base-MLX-fp16

sourceHugging Facecc-by-nc-4.0updated 3mo agoView on Hugging Face

0likes49downloads

Model Card

F5-TTS v1 Base — MLX fp16

Native MLX bundle of F5-TTS v1 Base — a 336M-parameter zero-shot voice-cloning TTS model using DiT flow matching with a Vocos vocoder and 24 kHz output. Give it a short reference clip plus its transcript and it speaks any English or Mandarin text in that voice.

Part of soniqo.audio — an on-device speech toolkit for Apple Silicon. Consumed by the open-source `speech-swift` library (module F5TTS).

Bundle size: 670 MB

Use cases

—Voice cloning — zero-shot cloning from reference audio + transcript.
—CLI reference — speech speak --engine f5 ... flags.
—Getting started — install speech-swift on macOS / iOS.

Capabilities

—Zero-shot voice cloning — one short reference clip plus its transcript, no fine-tuning
—24 kHz output decoded by Vocos
—Deterministic sampling — seeded flow matching, repeatable output
—Sampling controls — flow steps, CFG strength, sway coefficient, speaking rate
—English + Mandarin input, including mixed EN/ZH text — hanzi go through the bundled pinyin lexicon (TONE3, tone sandhi baked in at export)

Model

Field	Value
Architecture	F5-TTS v1 Base, DiT flow matching
Parameters	336M class
Vocoder	Vocos mel 24 kHz
Sample rate	24 kHz
Languages	English + Mandarin Chinese (mixed text supported)
Precision	fp16 reference bundle
Runtime	native Swift/MLX synthesis in `speech-swift`

Files

File	Description
`model.safetensors`	F5 CFM/DiT EMA weights, `ema_model.` prefix stripped, fp16
`vocos.safetensors`	Vocos vocoder weights converted from PyTorch `.bin`
`vocab.txt`	Upstream tokenizer vocabulary
`pinyin_lexicon.tsv`	Mandarin frontend lexicon (TONE3, sandhi baked; pypinyin + jieba data, MIT)
`vocos_config.yaml`	Upstream Vocos config
`config.json`	Architecture, sample-rate, and license metadata

Precision

No quantization. This is the fp16 reference bundle. Quantized variants are only promoted after passing the same cloned-voice ASR roundtrip gate.

Usage with `speech-swift`

swift

import F5TTS

let model = try await F5TTSModel.fromPretrained(
    modelId: "aufklarer/F5TTS-v1-Base-MLX-fp16"
)
let audio = try await model.generate(
    text: "This is a short local voice cloning test.",
    referenceAudio: URL(fileURLWithPath: "reference.wav"),
    referenceText: "The words spoken in the reference recording."
)

Or via the CLI:

bash

speech speak "This is a short local voice cloning test." --engine f5 \
  --voice-sample reference.wav \
  --f5-reference-text "The words spoken in the reference recording." \
  -o cloned.wav

Source

This bundle is converted from the upstream PyTorch weights at SWivid/F5-TTS (checkpoint F5TTS_v1_Base/model_1250000.safetensors), paired with the charactr/vocos-mel-24khz vocoder. Paper: F5-TTS.

License

CC-BY-NC 4.0 — inherited from the released upstream weights (Emilia-trained). This bundle is non-commercial/research unless a commercial-safe checkpoint is substituted.

Responsible use

Voice cloning capability is included. Users are responsible for obtaining consent for any voice that is cloned and for not using the model to impersonate individuals without their permission, generate disinformation, or commit fraud.