mlr2000/vocoder-large

VocBulwark HiFi-GAN — watermarking neural vocoder (inference export)

Speaker-conditioned BigVGAN / HiFi-GAN neural vocoder. It turns an input mel-spectrogram (what is said) into a 24 kHz waveform, conditioned on a precomputed 768-d speaker embedding (whose voice). Every clip it generates carries a fixed 50-bit provenance watermark that identifies this specific model instance, see Watermark.

This is the lean, inference-only vocoder: the frozen perceptual-loss base models (Whisper / WavLM / Wav2Vec2), the training discriminators, and the speaker encoder have all been stripped, you pass the speaker embedding in. Use the companion speaker-encoder repo to turn a reference clip into that embedding. The modeling code is bundled, so it loads with trust_remote_code=True without the training repo.

Model summary

Companion Models

This model is part of a set of 6 repositories:

Usage

import torch
from transformers import AutoModel

model = AutoModel.from_pretrained("mlr2000/vocoder-large", trust_remote_code=True).eval()

mel = torch.randn(1, model.config.hifigan_in_channels, 200)      # [B, mel, T]
emb = torch.randn(1, model.config.speaker_embedding_size)        # [B, 768] from the speaker encoder
with torch.no_grad():
    audio = model(mel_spectrogram=mel, speaker_embedding=emb).audio
# audio: [B, 1, samples] @ model.config.target_sample_rate

See `example_roundtrip.ipynb` in this repo for the full pipeline (reference clip → speaker encoder → embedding → vocode → verify watermark).

Training

Watermark

Every clip this model generates carries a fixed 50-bit provenance watermark (config.fixed_watermark) that identifies this specific model instance. It is embedded automatically inside forward and cannot be disabled or changed through this interface — there is deliberately no watermark argument to override.

To verify whether a given audio clip was generated by this model, use the companion detector repo (mlr2000/vocoder-large-watermark-detector), which extracts the embedded bits and compares them to the known fixed code.

Notes

• —Inputs: log-mel spectrogram (config.hifigan_in_channels channels) and a [B, config.speaker_embedding_size] speaker embedding.
• —Output: mono waveform at config.target_sample_rate.
• —Use the embedding from the speaker encoder this vocoder was trained with, a mismatched encoder will not condition it correctly.
• —Not intended for voice cloning of real individuals without consent, or any deceptive / impersonation use.

Citation

If you use this model, please cite:

@misc{muletta2026,
  title  = {Training a Discriminator-Free Foundation Vocoder 
             with Integrated Audio Watermarking},
  author = {Muletta, Romolo and Deriu, Jan},
  year   = {2026},
  note   = {VT2 Project Report, ZHAW School of Engineering}
}

License

cc-by-4.0. Trained on MLS (CC-BY-4.0) and Common Voice (CC0); builds on BigVGAN (MIT) and wav2vec 2.0 (Apache-2.0). Please retain attribution when redistributing or building on this model.