CoolFace
Modelpublic

appautomaton/step-audio-editx-8bit-mlx

sourceHugging Faceapache-2.0updated 14d agoView on Hugging Face
2likes147downloads
Model Card

Step-Audio-EditX — MLX 8-bit

This repository contains a self-contained pure-MLX int8 conversion of Step-Audio-EditX for local voice cloning and expressive audio editing on Apple Silicon. All pipeline components are stored as .safetensors — no PyTorch, ONNX, or NumPy files are required at inference time.

Model Details

  • —Developed by: AppAutomaton
  • —Upstream model: `stepfun-ai/Step-Audio-EditX`
  • —Task: zero-shot voice cloning, expressive audio editing
  • —Runtime: MLX on Apple Silicon
  • —Precision: int8 for Step1 LM, Flow model, and VQ02 tokenizer; bf16 for the rest
  • —Total size: ~4.1 GB (down from ~7.7 GB upstream)

Bundle Contents

This bundle is self-contained — all weights are packaged in one repository.

FileComponentFormatSize
model.safetensorsStep1 LM (3.5B params)int83.5 GB
flow-model.safetensorsFlow model (DiT + conformer)int8181 MB
vq02.safetensorsVQ02 audio tokenizerint8162 MB
vq06.safetensorsVQ06 audio tokenizerbf16249 MB
hift.safetensorsHiFT vocoderbf1640 MB
campplus.safetensorsCampPlus speaker embeddingbf1613 MB
flow-conditioner.safetensorsFlow conditionerbf162.5 MB
config.jsonStep1 LM config + quantizationJSON—
flow-model-config.jsonFlow model configJSON—
vq02-config.json, vq06-config.jsonTokenizer configsJSON—
step-audio-tokenizer-assets.safetensorsVQ02 codebook + CMVNFP32~2 MB
step-audio-tokenizer-config.jsonTokenizer runtime configJSON—
frontend-config.jsonPrompt mel frontend configJSON—
hift-config.json, campplus-config.json, flow-conditioner-config.jsonComponent configsJSON—
tokenizer.json, tokenizer_config.jsonStep1 tokenizerJSON—

How to Get Started

Download the bundle:

bash
hf download appautomaton/step-audio-editx-8bit-mlx \
  --local-dir models/stepfun/step_audio_editx/mlx-int8

Voice cloning:

bash
python scripts/generate/step_audio_editx.py \
  --prompt-audio reference.wav \
  --prompt-text "Transcript of reference audio." \
  -o cloned.wav \
  clone --target-text "New speech in the cloned voice."

Audio editing (change emotion):

bash
python scripts/generate/step_audio_editx.py \
  --prompt-audio input.wav \
  --prompt-text "Transcript of input audio." \
  -o happy.wav \
  edit --edit-type emotion --edit-info happy

Supported Edit Types

Edit typeDescription`--edit-info` examples
emotionChange the emotion of speechhappy, sad, angry, surprised
styleChange speaking stylewhispering, broadcasting, formal
speedChange speaking speedfast, slow
denoiseRemove noise from audionot used
vadRemove silences from audionot used
paralinguisticAdd non-verbal soundsrequires --target-text

Architecture

Five-stage pipeline, all running pure MLX with bf16 activations:

  1. 1.Step1 LM (3.5B params, int8) — autoregressive dual-codebook token generation
  2. 2.CampPlus (bf16) — speaker embedding extraction from reference audio
  3. 3.Flow conditioner (bf16) — conditions generation on speaker embedding
  4. 4.Flow model (int8) — flow-matching mel spectrogram generation
  5. 5.HiFT vocoder (bf16) — mel spectrogram to waveform

The VQ02 and VQ06 tokenizers encode reference audio into dual codebook tokens consumed by Step1.

Performance

On Apple Silicon with int8 weights and bf16 activations, real-time factor (RTF) is approximately 1.46x for voice cloning — faster than real-time.

Links

License

Apache 2.0 — following the upstream license published with `stepfun-ai/Step-Audio-EditX`.