notmax123/QwenTTS-he-1.7B

QwenTTS-he-1.7B

A Hebrew LoRA adapter for Qwen3-TTS-12Hz-1.7B-Base, trained on ~131k Hebrew utterances with stressed-IPA text conditioning.

The adapter adds Hebrew to the base model without touching a single base weight. Load it and you get Hebrew; disable it and you get the original model, bit-for-bit, with all ten of its native languages intact.

Trained by Max Melichov.

What this is

Trained parameters

LoRA is applied to q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj across both the talker backbone and the MTP code_predictor.

These modules are additionally trained in full and shipped inside the adapter (modules_to_save), because Hebrew phonotactics need the output distribution itself to move, not just the attention/MLP deltas:

• —codec_head
• —text_projection
• —lm_head.0 … lm_head.14 (the 15 residual-codebook heads)

The text embedding table is not resized or retrained, so the tokenizer and vocabulary are unchanged from the base model.

Important: the text field takes IPA, not Hebrew script

This adapter was trained with stressed IPA in the text field. Hebrew orthography is famously under-specified for vowels, so feeding raw Hebrew script will not work well. Convert first with a Hebrew G2P (the training data used RenikudPlus for diacritization followed by IPA conversion).

Hebrew:  עוֹד רֶגַע אֲנִי אֶתְיַיחֵס לִיהוּדָה וְשׁוֹמְרוֹן.
IPA:     ʔˈod ʁˈeɡa ʔanˈi ʔetjaχˈes lihudˈa veʃomʁˈon.

The ˈ (U+02C8) primary-stress marker precedes the stressed vowel and matters — the model learned prosody from it.

Usage

Requires the Qwen3-TTS inference code plus peft. The adapter wraps the model's `talker` submodule, not the whole model.

import torch, soundfile as sf
from peft import PeftModel
from qwen_tts import Qwen3TTSModel

tts = Qwen3TTSModel.from_pretrained(
    "Qwen/Qwen3-TTS-12Hz-1.7B-Base",
    device_map="cuda:0",
    dtype=torch.bfloat16,
    attn_implementation="sdpa",
)

# Attach the Hebrew adapter to the talker.
tts.model.talker = PeftModel.from_pretrained(
    tts.model.talker, "notmax123/QwenTTS-he-1.7B"
)
tts.model.eval()

wavs, sr = tts.generate_voice_clone(
    text="ʃalˈom, mˈa ʃlomχˈa hajˈom?",   # stressed IPA
    language="Auto",
    ref_audio="my_reference_voice.wav",
    x_vector_only_mode=True,
    non_streaming_mode=True,
    do_sample=False,
    subtalker_dosample=False,
    repetition_penalty=1.0,
    max_new_tokens=2048,
)
sf.write("out.wav", wavs[0], sr)

language="Auto" is what the model was trained and sampled with — Hebrew is not in the base model's language table, and the IPA text carries the phonetics.

Getting the base model's other languages back

Do not run Chinese / English / French / German / Italian / Japanese / Korean / Portuguese / Russian / Spanish with the adapter active — it shifts the output distribution toward Hebrew. Wrap those calls instead:

with tts.model.talker.disable_adapter():
    wavs, sr = tts.generate_voice_clone(
        text="Hello, how are you today?",
        language="English",
        ref_audio=ref,
        ref_text=ref_text,
    )

With the adapter disabled the forward pass is bit-identical to the unmodified base model (verified: all 404 base tensors unchanged, and generated audio matches the pre-adapter base at max|diff| = 0 for English and Japanese).

Do not merge

merge_and_unload() bakes the Hebrew deltas into the base weights permanently and destroys the guarantee above. Keep the adapter separate and toggle it.

Training

Eval loss decreased monotonically at every 500-step checkpoint, from 2.2999 (step 500) to 2.1339 (step 6000) — the run had not yet plateaued, so more steps would likely still help.

<details> <summary>Eval loss by step</summary>

</details>

Files

Limitations

• —IPA input required. Raw Hebrew text needs a G2P pass first.
• —Trained on read/narrated speech; expressive or conversational Hebrew is out of distribution.
• —Speaker coverage comes from the training corpus's voices; cloning to a very different voice may carry over training-speaker prosody.
• —Hebrew only. Yiddish was deliberately excluded from this run.

License

Apache-2.0, matching the base model. See the Qwen3-TTS model card for base-model terms.

Citation

@misc{melichov2026qwentts-he,
  author = {Max Melichov},
  title  = {QwenTTS-he-1.7B: A Hebrew LoRA adapter for Qwen3-TTS-12Hz-1.7B-Base},
  year   = {2026},
  url    = {https://huggingface.co/notmax123/QwenTTS-he-1.7B}
}