audio-cpp/LiveAvatar-GGUF

LiveAvatar GGUF for audio.cpp

Native GGUF package of Wan-AI/Wan2.2-S2V-14B with the official Quark-Vision/Live-Avatar LoRA merged for audio.cpp. LiveAvatar generates an audio-driven avatar video from a reference image, speech audio, and a text description.

Files

All three files are required.

Low-VRAM mode

LiveAvatar can keep the denoiser transformer blocks in pinned host memory and stage one layer group at a time. This makes 720p generation possible near a 16 GiB VRAM limit, at the cost of additional host-to-device transfers.

The validated 1280x720 run generated 93 frames at 16 FPS (5.81 seconds of video) in 276 seconds wall time (4 minutes 36 seconds) and peaked at 15,993 MiB VRAM on an RTX 5090.

Weight streaming is slower than the normal resident-weight path. In a matched 240p comparison, model time increased from 13.33 to 21.73 seconds, or about 1.63x slower. The exact slowdown depends on resolution, hardware, and host memory bandwidth.

Full-duration 720p example

This example uses the official LiveAvatar Cyclops Baker image and speech audio. The official reference image is only 720x400, so this example also shows 1280x720 generation from a lower-resolution image condition. The complete 11.6-second WAV is retained in the generated video.

[image]

<video controls playsinline preload="metadata" width="100%" src="https://huggingface.co/audio-cpp/LiveAvatar-GGUF/resolve/main/examples/cyclops/output-720p-weight-streaming.mp4"></video>

Download the generated video

The run used the low-VRAM controls above with denoiser weight streaming, layerwise batch size 16, one target-cache block, F16 VAE cache, VAE encoder chunk size 4, and VAE decoder tile size 320. Measurements were collected with the CUDA debug build on an NVIDIA GeForce RTX 5090.

480p cache trade-off

These two videos use the same Cyclops Baker inputs, prompt, seed, four clips, and complete 11.6-second audio. Only target_cache_blocks changes. A longer cache retains more temporal context from preceding video blocks and can improve continuity across clip boundaries, but it does not directly increase per-frame resolution or detail.

<table> <tr> <td width="50%"><strong>Two cache blocks</strong><br><video controls playsinline preload="metadata" width="100%" src="https://huggingface.co/audio-cpp/LiveAvatar-GGUF/resolve/main/examples/cyclops/output-480p-cache2.mp4"></video><br><code>targetcacheblocks=2</code></td> <td width="50%"><strong>Three cache blocks</strong><br><video controls playsinline preload="metadata" width="100%" src="https://huggingface.co/audio-cpp/LiveAvatar-GGUF/resolve/main/examples/cyclops/output-480p-cache3.mp4"></video><br><code>targetcacheblocks=3</code></td> </tr> </table>

480p is the better starting point for prompt, identity, motion, and cache-window experiments. Iterate at 480p, select the strongest result, and upscale that video afterward instead of paying the 720p generation cost for every attempt.

Quality comparison

Each clip uses the same reference image, audio, prompt, seed 420, four Euler steps, guidance 0, scheduler shift 3, 416x240 resolution, 16 FPS, and 84 requested frames (81 output frames, 5.06 seconds). The reference uses SageAttention, memory_saver=true, full target cache, F32 VAE cache, no VAE tiling, and non-layerwise denoising. Each other cell changes only the parameter shown below the video.

Inputs: reference speech and reference image.

Prompt: A stout, cheerful dwarf with a magnificent braided beard adorned with metal rings, wearing a heavy leather apron. He is standing in his fiery, cluttered forge, laughing heartily as he explains the mastery of his craft, holding up a glowing hammer. Style of Blizzard Entertainment cinematics, warm, dynamic lighting from the forge.

Measured with the CUDA debug build on an NVIDIA GeForce RTX 5090. Wall time includes model loading and output generation. Peak VRAM is total device memory used during the run.

<table> <tr> <td width="50%"><strong>Reference</strong><br><video controls playsinline preload="metadata" width="100%" src="https://huggingface.co/audio-cpp/LiveAvatar-GGUF/resolve/main/examples/quality/reference.mp4"></video><br><code>reference controls</code></td> <td width="50%"><strong>FlashAttention</strong><br><video controls playsinline preload="metadata" width="100%" src="https://huggingface.co/audio-cpp/LiveAvatar-GGUF/resolve/main/examples/quality/flash-attention.mp4"></video><br><code>sageattention=false</code></td> </tr> <tr> <td width="50%"><strong>F16 VAE cache (current default)</strong><br><video controls playsinline preload="metadata" width="100%" src="https://huggingface.co/audio-cpp/LiveAvatar-GGUF/resolve/main/examples/quality/vae-cache-f16.mp4"></video><br><code>vaecachef16=true</code></td> <td width="50%"><strong>VAE decoder tiling</strong><br><video controls playsinline preload="metadata" width="100%" src="https://huggingface.co/audio-cpp/LiveAvatar-GGUF/resolve/main/examples/quality/vae-tile-320.mp4"></video><br><code>vaedecodertilesize=320</code></td> </tr> <tr> <td width="50%"><strong>Target cache: 1 block</strong><br><video controls playsinline preload="metadata" width="100%" src="https://huggingface.co/audio-cpp/LiveAvatar-GGUF/resolve/main/examples/quality/target-cache-1.mp4"></video><br><code>targetcacheblocks=1</code></td> <td width="50%"><strong>Target cache: 4 blocks</strong><br><video controls playsinline preload="metadata" width="100%" src="https://huggingface.co/audio-cpp/LiveAvatar-GGUF/resolve/main/examples/quality/target-cache-4.mp4"></video><br><code>targetcacheblocks=4</code></td> </tr> <tr> <td width="50%"><strong>Layerwise denoising</strong><br><video controls playsinline preload="metadata" width="100%" src="https://huggingface.co/audio-cpp/LiveAvatar-GGUF/resolve/main/examples/quality/layerwise-16.mp4"></video><br><code>denoiserlayerwise=true</code>, <code>denoiserlayerwise_batch=16</code></td> <td width="50%"></td> </tr> </table>

Run

audiocpp_cli \
  --task gen \
  --family liveavatar \
  --model /path/to/LiveAvatar-GGUF \
  --backend cuda \
  --threads 8 \
  --audio /path/to/reference.wav \
  --text "A detailed description of the speaker and scene." \
  --request-option generation_mode=liveavatar \
  --request-option reference_image_path=/path/to/reference.jpg \
  --request-option height=240 \
  --request-option width=416 \
  --out-dir outputs/liveavatar \
  --log

The default LiveAvatar configuration uses four Euler steps, scheduler shift 3, guidance scale 0, seed 420, 48 frames per clip, SageAttention, and 16 FPS.

For lower VRAM use, add:

--session-option liveavatar.denoiser_weight_streaming=true \
--request-option denoiser_layerwise=true \
--request-option denoiser_layerwise_batch=16

See the audio.cpp LiveAvatar documentation for resolution, duration, and memory-control options.

License

LiveAvatar and its Wan2.2 base model are released under the Apache License 2.0. This repository includes the upstream license in LICENSE. The GGUF conversion is a packaging format for audio.cpp and is not an official upstream release.