yang-qi1222/cosyvoice3_stream.cpp

cosyvoice3_stream.cpp

Runtime assets for cosyvoice3_yq_cpp, an inference-only CosyVoice3 C++/GGML streaming runtime.

This repository contains one project-compressed GGUF model, the two ONNX models needed to extract a voice prompt from reference audio, and ten small precomputed prompt features for immediate testing. Source code is distributed separately on Gitee.

Files

Every binary is pinned in SHA256SUMS. The source repository also ships a machine-readable manifest used by its downloader.

Model Lineage

The base model is FunAudioLLM/Fun-CosyVoice3-0.5B-2512. The published GGUF applies the following project-specific deployment changes:

• —a 12-layer student speech LLM obtained through structured layer pruning and distillation;
• —Flow inference distilled from 10 steps to 4 steps;
• —GGUF conversion with inner LLM MLP tensors quantized to Q8 and Flow/HiFT kept in F16.

The LLM compression workflow was informed by SPADE, with additional material on the SPADE project page. This is an independent CosyVoice3 engineering adaptation, not an official SPADE model or a claim that the paper's reported metrics transfer to this model.

Prompt Features

control10_01 through control10_10 are features extracted from synthetic teacher-model outputs used in this project's five-male/five-female Control10 evaluation. No source WAV files are distributed, and these prompts are not presented as the voice of any real person.

Use a prompt directly for the lowest startup overhead. To clone a new authorized reference voice, use speech_tokenizer_v3.onnx and campplus.onnx with the audio-enabled runtime to create a new prompt_speech.gguf.

Download And Run

git clone https://gitee.com/yang-qi1222/cosyvoice3_yq_cpp.git
cd cosyvoice3_yq_cpp
python3 -m pip install -r requirements-tools.txt
python3 scripts/download_assets.py \
  --manifest manifests/assets.example.json \
  --asset-root assets

Build and start the feature-only CPU server:

scripts/build_runtime.sh \
  --mode feature \
  --backend cpu \
  --build-dir build/feature-cpu

scripts/start_server.sh \
  --build-dir build/feature-cpu \
  --backend cpu \
  --model assets/models/student12_mlp_inner_q8_flow4_hift_f16.gguf \
  --prompt-speech assets/prompts/control10_01.gguf \
  --voice control10_01 \
  --port 8080

CUDA architecture, thread count, chunk size, and FlashAttention settings must be selected and validated for the deployment machine. See the source repository for the audio-input build and complete server examples.

Measured Baseline

The accepted project baseline covers only an NVIDIA RTX 5880 Ada Generation GPU, one serial request at a time, 16 host threads, chunk_tokens=75, LLM FlashAttention disabled, and Flow FlashAttention enabled.

These measurements must not be extrapolated to RTX 4060, Jetson AGX, other GPU architectures, concurrent service, or long-context synthesis.

Limitations

• —This repository distributes inference assets, not training checkpoints or training code.
• —The reference-audio path also requires an ONNX Runtime C/C++ SDK at build time; the two ONNX model files do not replace that dependency.
• —Voice cloning must only be performed with appropriate speaker consent and data rights.
• —The project is community maintained and is not affiliated with or endorsed by the official CosyVoice or SPADE teams.

License And Acknowledgements

Model assets are distributed under Apache-2.0. The C++ source repository uses the MIT license; source and model licenses are separate.

This work depends on and thanks:

• —QwenAudio/CosyVoice and the CosyVoice3 paper;
• —Lourdle/cosyvoice.cpp;
• —SPADE;
• —ggml, llama.cpp, and ONNX Runtime.

Please cite the original CosyVoice3 and SPADE papers when this model is used in research comparisons.