BricksDisplay/chatterbox-multilingual-ONNX-q4

Chatterbox Multilingual TTS - Q4 Quantized ONNX

Q4 weight-only quantized version of onnx-community/chatterbox-multilingual-ONNX for use with Transformers.js and ONNX Runtime Web.

Key Features

• —75% smaller: 790 MB vs 3.2 GB original
• —Single-file ONNX: No external data files, compatible with Transformers.js
• —Same quality: Minimal quality loss from Q4 quantization
• —23 languages supported: ar, da, de, el, en, es, fi, fr, he, hi, it, ja, ko, ms, nl, no, pl, pt, ru, sv, sw, tr, zh

Model Sizes

Usage

With ONNX Runtime (Python)

import onnxruntime

# Load Q4 models - single files, no external data needed
speech_encoder = onnxruntime.InferenceSession("onnx/speech_encoder.onnx")
embed_tokens = onnxruntime.InferenceSession("onnx/embed_tokens.onnx")
language_model = onnxruntime.InferenceSession("onnx/language_model.onnx")
conditional_decoder = onnxruntime.InferenceSession("onnx/conditional_decoder.onnx")

With Transformers.js (JavaScript)

// Models are single-file ONNX format, compatible with ONNX Runtime Web
import { AutoTokenizer } from '@huggingface/transformers';

const tokenizer = await AutoTokenizer.from_pretrained('ipsilondev/chatterbox-multilingual-ONNX-q4');

Quantization Details

• —Method: Q4 weight-only quantization using MatMulNBitsQuantizer
• —Block size: 32
• —Symmetric: Yes
• —Format: Single-file ONNX (no external data) for web compatibility

Important Parameters

When using these models, ensure you use the correct parameters:

repetition_penalty = 1.2  # CRITICAL: Do NOT use 2.0 - causes infinite loops
temperature = 0.8
top_p = 0.95
min_p = 0.05

Supported Languages

Credits

• —Original model: onnx-community/chatterbox-multilingual-ONNX
• —Base model: ResembleAI/chatterbox
• —Quantization by: ipsilondev

License

MIT License (same as original model)