CoolFace
Modelpublic

aguken-ai/Qwen-3-TTS-12Hz-0.6B-Base-hi-LoRA-Finetuned-BNB-NF4

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face
10likes99downloads
Model Card

Qwen3-TTS-12Hz-0.6B-Base · Hindi LoRA Finetune (BitsAndBytes NF4)

A Hindi (`hi`) LoRA finetune of `Qwen/Qwen3-TTS-12Hz-0.6B-Base`, with the LoRA adapter merged into the base weights and the result quantized to 4-bit NF4 using bitsandbytes.

The base Qwen3-TTS family ships with 10 built-in languages (Chinese, English, Japanese, Korean, German, French, Russian, Portuguese, Spanish, Italian) but does not include Hindi. This checkpoint extends the base model toward Hindi speech synthesis and voice cloning while keeping the memory footprint small enough to run on consumer GPUs.

Model Details

Base modelQwen/Qwen3-TTS-12Hz-0.6B-Base
ArchitectureQwen3TTSForConditionalGeneration (discrete multi-codebook LM talker + code predictor)
Parameters~0.6B (talker), plus speech tokenizer
Target languageHindi (hi)
Finetuning methodLoRA (adapter merged into base)
Quantizationbitsandbytes 4-bit NF4, double quantization, bfloat16 compute dtype
Speech tokenizerQwen3-TTS-Tokenizer-12Hz (12.5 Hz frame rate, 24 kHz audio)
Sample rate24 kHz
LicenseApache-2.0 (inherited from base)

The following modules are kept in higher precision (not quantized) for output quality: text_projection, codec_head, code_predictor, speaker_encoder.

Repository Contents

config.json                 # Model + quantization config
generation_config.json      # Default sampling params
model.safetensors           # 4-bit NF4 quantized weights (~1.2 GB)
merges.txt / vocab.json     # Text tokenizer
tokenizer_config.json
preprocessor_config.json
speech_tokenizer/           # Qwen3-TTS-Tokenizer-12Hz (encode/decode audio)

Installation

bash
conda create -n qwen3-tts python=3.12 -y
conda activate qwen3-tts

pip install -U qwen-tts
pip install -U bitsandbytes        # required to load the 4-bit weights
# Optional, for faster inference on supported GPUs:
pip install -U flash-attn --no-build-isolation

Usage

The quantization settings are stored in config.json, so the 4-bit weights load automatically — no extra BitsAndBytesConfig is needed.

Voice cloning (Hindi)

python
import torch
import soundfile as sf
from qwen_tts import Qwen3TTSModel

MODEL_ID = "<your-username>/Qwen-3-TTS-12Hz-Base-hi-LoRA-Finetuned-BNB-NF4"
# or a local path to this folder

model = Qwen3TTSModel.from_pretrained(
    MODEL_ID,
    device_map="cuda:0",
    attn_implementation="flash_attention_2",  # drop if flash-attn is not installed
)

# Short Hindi reference clip + its transcript for cloning
ref_audio = "path/to/reference_hindi.wav"
ref_text  = "नमस्ते, मेरा नाम आरव है और मुझे संगीत सुनना बहुत पसंद है।"

wavs, sr = model.generate_voice_clone(
    text="आज मौसम बहुत सुहाना है, चलिए थोड़ी देर बाहर टहलने चलते हैं।",
    language="Hindi",
    ref_audio=ref_audio,
    ref_text=ref_text,
)

sf.write("output_hindi_clone.wav", wavs[0], sr)

Loading with Transformers directly

python
import torch
from transformers import AutoModelForCausalLM, AutoProcessor

model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    device_map="cuda:0",
    trust_remote_code=True,
)
processor = AutoProcessor.from_pretrained(MODEL_ID, trust_remote_code=True)

Finetuning Details

<!-- TODO: fill in once finalized -->

MethodLoRA, adapter merged into base weights
Target languageHindi (hi)
LoRA rank / alpha / dropoutTBD
Target modulesTBD
Training dataTBD
Epochs / stepsTBD
Optimizer / LR / scheduleTBD
HardwareTBD

Acknowledgements & Citation

This work builds on Qwen3-TTS by the Qwen team at Alibaba Cloud.

bibtex
@article{Qwen3-TTS,
  title={Qwen3-TTS Technical Report},
  author={Hangrui Hu and Xinfa Zhu and Ting He and Dake Guo and Bin Zhang and Xiong Wang and Zhifang Guo and Ziyue Jiang and Hongkun Hao and Zishan Guo and Xinyu Zhang and Pei Zhang and Baosong Yang and Jin Xu and Jingren Zhou and Junyang Lin},
  journal={arXiv preprint arXiv:2601.15621},
  year={2026}
}

References