Modelpublic

TigreGotico/voiceclonnx-openvoice-v2

sourceHugging Facemitupdated 4mo agoView on Hugging Face

0likes54downloads

Model Card

vconnx-openvoice-v2

ONNX export of the OpenVoice v2 tone-color converter (myshell-ai/OpenVoice, MIT license) for use with vconnx — a pure-ONNX zero-shot voice conversion toolkit.

Export details

Both components are exported from the upstream SynthesizerTrn (myshell-ai/OpenVoice) with strict state-dict loading — no architecture reconstruction. The checkpoint loads with 0 missing keys and 0 unexpected keys.

Component	File	Size
Reference encoder (FP32)	`tone_ref_encoder.onnx`	3.1 MB
Reference encoder (INT8)	`tone_ref_encoder_q8.onnx`	2.2 MB
Voice converter (FP32)	`tone_converter.onnx`	122.1 MB
Voice converter (INT8)	`tone_converter_q8.onnx`	38.9 MB

Architecture

Sub-graph	ONNX inputs	ONNX output
`tone_ref_encoder.onnx`	`spec` `(B, T, 513)` float32 — linear STFT magnitude	`tone_embedding` `(B, 256)`
`tone_converter.onnx`	`spec` `(B, 513, T)`, `spec_lengths` `(B,)`, `src_g` `(B, 256, 1)`, `tgt_g` `(B, 256, 1)`	`audio` `(B, 1, samples)` float32 — raw waveform

The converter includes the full VITS-style flow decoder and HiFi-GAN vocoder; it outputs raw audio directly. No separate vocoder step is needed at inference.

Preprocessing: linear magnitude spectrogram matching upstream spectrogram_torch — Hann window, n_fft=1024, hop=256, win=1024, reflect-pad 384 on each side, sqrt(Re² + Im² + 1e-6). No log compression.

Parity (upstream torch vs ONNX)

Component	max_abs_delta	mean_abs_delta	Status
`tone_ref_encoder`	8.64e-07	2.45e-07	PASS
`tone_converter` (5 seeds)	1.08e-02 (worst)	1.26e-04 (avg)	PASS

The converter max_abs divergence is due to float32 accumulation through 4 residual coupling blocks in the flow — the quality-relevant metric is mean_abs, which passes at 1e-3.

E2E sanity check

Converted a 2 s synthetic source (220 Hz harmonics) to a 330 Hz reference:

Metric	Value
Output duration	1.997 s (source 2.000 s, ratio 0.998)
RMS	0.268
Spectral flatness	0.072 (tonal, not noise)
Sample rate	22050 Hz

Usage

python

from vconnx import VoiceCloner

cloner = VoiceCloner(engine="openvoice")
cloner.clone_voice("source.wav", "reference.wav", "output.wav")

Or with the low-level adapter:

python

from vconnx.engines.openvoice import OpenVoiceV2Adapter

adapter = OpenVoiceV2Adapter(quantized=False)
adapter.clone_voice("source.wav", "reference.wav", "output.wav")

Install with: pip install vconnx[openvoice]

Provenance

—Upstream weights: myshell-ai/OpenVoiceV2
—Upstream source: myshell-ai/OpenVoice
—License: MIT ("Starting from April 2024, both V2 and V1 are released under MIT License. Free for commercial use." — official README)
—Export method: legacy TorchScript ONNX exporter (dynamo=False), opset 14 (new dynamo exporter fails on GRU)
—Strict load: 0 missing keys, 0 unexpected keys