AMAImedia/NOESIS-Qwen3-TTS-1.7B-Talker-Combo-LLM2-ASR2-BF16

<!-- apex-banner --> <div style="background-color: #f59e0b; color: white; padding: 20px; border-radius: 10px; text-align: center; margin: 20px 0;"> <h2 style="color: white; margin: 0 0 10px 0;">⚡ Each donation funds the next large quant.</h2> <p style="font-size: 16px; margin: 0 0 15px 0;">I host free GGUF or MoE quants as independent research.<br> Local hardware: <b>Mechrevo Kuangshi GM7AG0M</b> — RTX 3060 Laptop 6GB GDDR6, 64GB DDR5, i7-12700H (14C/20T, 4.7GHz), Windows 11, Samsung 990 Pro.<br> Good for imatrix and 0.6–35B-class work in RAM. <b>9B+ and searches need rented H200/Blackwell</b>, typically $100 per quant.</p> <p style="font-size: 20px; margin: 0;"> <a href="https://boosty.to/amaimediacom" style="color: white; text-decoration: underline;">🎉 Boosty🦄</a>  |  <a href="https://donatex.gg/widgets/donation-goal/89bc59e8-b69c-467a-9aa0-5e1c3f8842d6" style="color: white; text-decoration: underline;">☕ Buy Me a Coffee🦄</a>  |  <a href="https://www.donationalerts.com/r/djbionicl" style="color: white; text-decoration: underline;">⭐ DonationAlerts🦄</a> </p> <p style="font-size: 14px; margin: 10px 0 0 0; opacity: 0.9;">💚 Thanks to Hugging Face for extra storage.🦄</p> </div>

NOESIS / AMAImedia

Released as part of the NOESIS Professional Multilingual Dubbing Automation Platform (framework: DHCF-FNO -- Deterministic Hybrid Control Framework for Frozen Neural Operators).

• —Founder: Ilia Bolotnikov
• —Organization: AMAImedia.com
• —X (Twitter): @AMAImediacom
• —LinkedIn: Ilia Bolotnikov
• —Telegram: @AMAImediacom
• —NOESIS version: v16.1
• —Release date: 2026-06-11; r16 adapter update 2026-08-28

[image]

NOESIS-Talker-Combo (Darwin cross-modal 1.7B, 112-language tested route)

Production 1.7B Talker bundle for the NOESIS Voice Router-Swap. The frozen Combo backbone combines the Qwen3-TTS-1.7B Talker with a small cross-modal FFN blend from Qwen3-1.7B-LLM and Mega-ASR. The supplied per-family LoRA adapters are part of the NOESIS 112-language tested route; they remain separate and are selected by family without merging all adapters into one destructive checkpoint.

This directory is a model-weight and adapter bundle, not a standalone GUI application. In production it is loaded by the parent NOESIS router and its resident Qwen TTS worker. The router selects one language family, applies one adapter at the prescribed scale, synthesizes the current segment, and swaps the adapter or model only when the target language changes.

NOESIS provenance

Darwin / Qwen lineage

This bundle is the NOESIS Darwin cross-modal 1.7B Talker line. The model history has two distinct stages: first, the 1.7B speech/text branches were combined in the Darwin merge; second, the resulting Talker route was expanded with NOESIS training and family adapters. Language expansion did not mean merging all LoRA adapters into one checkpoint.

The five 1.7B branches above must not be confused with five independent public Qwen repositories: the original Qwen Base, LLM and ASR checkpoints are public links; Talker/Combo are the speech and derived cross-modal branches represented by the Qwen3-TTS/Darwin lineage and this NOESIS artifact. The model card keeps this distinction explicit for reproducibility and attribution.

Language and test accounting

The canonical NOESIS claim for this bundle is 112 languages tested end-to-end (112/112). The 95-code family-adapter table below is a routing table for the documented family adapters, not the complete count of the NOESIS test surface. The legacy file named manifest_94.json is a partial adapter-sweep manifest and must not be interpreted as the total language coverage. The upstream Qwen3-TTS codec itself has 10 native language IDs; the remaining languages use the NOESIS family, sibling, or generic route selected by the production router.

What is in the bundle

NOESIS-Qwen3-TTS-1.7B-Talker-Combo-LLM2-ASR2-BF16/
├── README.md
├── LICENSE
├── model.safetensors             # NOESIS-Qwen3-TTS-1.7B-Talker-Combo-LLM2-ASR2-BF16
├── config.json
├── generation_config.json
├── tokenizer_config.json
├── vocab.json / merges.txt
├── speech_tokenizer/              # Qwen3-TTS audio tokenizer assets
├── adapters/                      # one family adapter at a time
└── test_audio_perfamily/          # evaluation references and manifests

The deprecated attempt to sum all eleven adapters into one merged checkpoint is not the production model. Merging unrelated family adapters destroyed EOS and codec behavior and produced long non-speech outputs. Always load the Combo base and select exactly one family adapter when a non-native route is required.

Language → family adapter

The ten native Qwen3-TTS languages -- en, ru, zh, de, fr, es, ja, ko, it, and pt -- normally use plain Combo without a family adapter. Arabic (ar) is routed through the semitic family adapter in this bundle. Other languages select the matching family, sibling, or generic route. The production router performs this mapping automatically.

r16 adapter update

The indic and iranian families have r16 variants and should use them when the directory is present. The router falls back to r8 if an r16 directory is unavailable. The semitic family remains on r8. Several Brahmic Indic languages—especially te, pa, ml, or, and kn—showed EOS overgeneration in the current sweep; increasing rank alone does not fix that behavior. They require a dedicated training and EOS-fix pass rather than repeated re-generation.

Quality status

The complete NOESIS test program covered all 112 languages (112/112 tested). The dated per-family adapter sweep is a subset of that program: its legacy manifest_94.json records 86 scored checks plus six no-FLEURS skips and is not a total-coverage manifest. The full 112-language result is the canonical coverage claim; each route must still pass the normal NOESIS ASR, speaker-similarity, emotion, and timing gates before production acceptance.

How to run in NOESIS production

There is intentionally no run.bat in this model directory. Do not launch the safetensors file directly. The supported route is the parent NOESIS runtime:

NOESIS-3.5B-A0.5B-DUBBING-FILM/
├── demo_server/_voice_swap_runner.py     # router / subprocess orchestration
├── NOESIS-Qwen3-Omni-MoE-BF16/
│   ├── _qwen_tts_worker.py              # resident JSON-IPC worker
│   └── NOESIS-Qwen3-TTS-1.7B-Talker-Combo-LLM2-ASR2-BF16/
│       ├── model.safetensors
│       └── adapters/
└── dubbing/                              # timing and quality pipeline

The worker runs inside the frozen _venv_qwen_tts_457 environment. On Windows, its subprocess must inherit PYTHONUTF8=1 and PYTHONIOENCODING=utf-8; otherwise non-ASCII text can corrupt the JSON IPC stream.

A production request contains the target text, the reference audio or speaker embedding, the target language, the selected family adapter, and the adapter scale:

{
  "op": "clone",
  "text": "Welcome to the Sanctuary, Miss Amelia.",
  "target_lang": "en",
  "adapter": "germanic",
  "adapter_scale": 0.32
}

The exact worker schema is owned by the parent router and may add reference-audio, x-vector, duration, and cache fields. The adapter policy does not change: one family adapter, one scale, one active Talker route. Do not send all family adapters in one request.

Typical Windows environment

cd /d B:\Downloads\Portable\NOESIS-VC-ONE\models\llm\NOESIS-3.5B-A0.5B-DUBBING-FILM
set PYTHONUTF8=1
set PYTHONIOENCODING=utf-8
set NOESIS_TTS_PRIMARY=combo

REM The parent production server starts the resident worker and routes requests.
REM Start it using the bundle's normal demo_server launcher; do not run the
REM model.safetensors file as a Python script.

NOESIS_TTS_PRIMARY=combo is an explicit fallback selection. The default generic production route is OmniVoice (omni06); set the variable only when testing or intentionally routing through Combo.

Direct model-loading outline

For development and diagnostics, load the base and one adapter through the Qwen3-TTS/PEFT runtime used by the parent worker. The following is an outline of the required policy, not a replacement for the pinned worker implementation:

# Pseudocode: use the parent worker's pinned loader and tokenizer setup.
base = load_qwen3_tts("NOESIS-Qwen3-TTS-1.7B-Talker-Combo-LLM2-ASR2-BF16")
base.load_lora("adapters/NOESIS-Talker-Combo-romance-LoRA-r8-BF16")
base.set_lora_scale(0.32)
waveform = base.generate_voice_clone(
    text="Bonjour, bienvenue au Sanctuaire.",
    reference_audio="clean_reference.wav",
    language="fr",
)
base.unload_lora()

The parent worker also handles CUDA placement, reference-text or x-vector requirements, output normalization, caching, and cleanup. Reimplementing those pieces outside the worker can create VRAM leaks or a Talker ICL failure.

Hardware and runtime notes

Failure handling

If a Talker clone job reports a misleading worker timeout, inspect the worker stderr first. A common underlying error is ValueError('ref_text required when x_vector_only_mode=False'). Provide the exact reference transcript or explicitly use x-vector-only mode when the selected route supports it. Do not diagnose this as a GPU timeout until the stderr log has been checked.

When a family adapter emits no EOS, very long non-speech, or a token-cap termination, reject the take and route to the designated OmniVoice or subtitle fallback. Re-running the same weights does not repair an under-trained adapter.

NOESIS sealed rules

• —`R-TALKER-112LANG-PER-FAMILY-SWAP-NOT-MERGE` — load Combo plus one family adapter and swap by language; never sum all family adapters.
• —`R-VOICE-ROUTER-SWAP-NOT-MERGE` — language specialists remain separate in the production router.
• —`R-ROUTER-LANG-SPLIT-1.7B-VS-0.6B` — Combo is the 1.7B route for designated main languages and fallback cases; OmniVoice is the primary generic 0.6B route.
• —`R-TALKER-EOS-GATE` — reject token-cap or no-EOS generations instead of passing non-speech downstream.
• —`R-REF-TEXT-REQUIRED-ICL` — provide ref_text for ICL cloning unless x-vector-only mode is explicitly selected.
• —`R-APACHE-CLEAN` — preserve the upstream Apache-2.0 license and NOESIS attribution.
• —`R-VENDORED-INTERNAL` — this bundle is loaded from the parent NOESIS production tree, not treated as an unrelated standalone checkpoint.

References

| 1 | Qwen3-TTS Talker 1.7B | Speech-generating Talker backbone in the Qwen3-TTS family. | | 2 | Qwen3-TTS Base 1.7B | Original TTS base scaffold: Qwen/Qwen3-TTS-12Hz-1.7B-Base. | | 3 | Qwen3 LLM 1.7B | Text-language-model branch: Qwen/Qwen3-1.7B. | | 4 | Qwen3 ASR 1.7B | Speech-understanding/ASR branch: Qwen/Qwen3-ASR-1.7B. |

License

Apache License 2.0. The Combo checkpoint derives from Qwen3-TTS-12Hz-1.7B-Base and the per-family adapters are NOESIS / AMAImedia LoRA additions. See `LICENSE`.

Detailed Launch Guide (EN)

This section is a practical guide: how to run the 1.7B Talker Combo with adapters, what is inside the _venv_qwen_tts_457 environment, which tool versions are required, and where everything is downloaded from.

1. Quick start (TL;DR)

REM 1) Enter the environment bundle folder
cd /d B:\Downloads\Portable\NOESIS-VC-ONE\models\llm\NOESIS-3.5B-A0.5B-DUBBING-FILM\NOESIS-Qwen3-Omni-MoE-BF16

REM 2) UTF-8 is mandatory for the JSON IPC (otherwise non-ASCII text corrupts the stream)
set PYTHONUTF8=1
set PYTHONIOENCODING=utf-8

REM 3) Start the resident worker (it keeps the model in VRAM between requests)
_venv_qwen_tts_457\Scripts\python.exe _venv_qwen_tts_457\_qwen_tts_worker.py

REM 4) The worker reads one JSON line from stdin and writes one JSON line to stdout.
REM    Example request (one line, then Enter):
REM    {"cmd":"clone","bundle":"NOESIS-Qwen3-TTS-1.7B-Talker-Combo-LLM2-ASR2-BF16","ref":"C:\\ref\\voice.wav","ref_text":"exact reference transcript","text":"Hello world","lang":"english","adapter":"adapters\\NOESIS-Talker-Combo-germanic-LoRA-r8-BF16","adapter_scale":0.32,"out":"C:\\out\\take.wav"}

Full worker path:

B:\Downloads\Portable\NOESIS-VC-ONE\models\llm\NOESIS-3.5B-A0.5B-DUBBING-FILM\NOESIS-Qwen3-Omni-MoE-BF16\
├── _venv_qwen_tts_457\                 <- ISOLATED ENVIRONMENT
│   ├── Scripts\python.exe              <- Python 3.11.14
│   ├── Lib\site-packages\              <- torch 2.7.1+cu128, transformers 4.57.3, qwen-tts 0.1.1 ...
│   ├── _qwen_tts_worker.py            <- resident JSON-IPC worker (THIS is what you launch)
│   └── _talker_duration_bias.py       <- TED-TTS EOS steering (A1, loaded by the worker)
└── NOESIS-Qwen3-TTS-1.7B-Talker-Combo-LLM2-ASR2-BF16\
    ├── model.safetensors               <- Combo 1.7B Talker (~3.6 GB, BF16)
    ├── config.json / generation_config.json / preprocessor_config.json
    ├── tokenizer_config.json / vocab.json / merges.txt
    ├── speech_tokenizer\               <- Qwen3-TTS 12 Hz audio tokenizer
    ├── adapters\                       <- 15 family LoRAs (one at a time)
    └── test_audio_perfamily\           <- evaluation reference wavs

2. The _venv_qwen_tts_457 environment — what is inside

Path: ...\NOESIS-Qwen3-Omni-MoE-BF16\_venv_qwen_tts_457

Exact package versions (verified with importlib.metadata)

WARNING about the transformers version. The Qwen3TTSForConditionalGeneration model requires exactly transformers 4.57.3 (see transformers_version in config.json). Any other 4.57.x version changes the behavior of generate_voice_clone and corrupts codec/EOS. Do not upgrade transformers in this venv without a fresh smoke test.

Known trampoline issue (important)

Scripts\*.exe (for example qwen-tts-demo.exe) are uv trampolines that can fail with:

error: uv trampoline failed to canonicalize script path

Cause: the uv base interpreter was moved/deleted. However Scripts\python.exe still works directly — use it.

The _qwen_tts_worker.py worker additionally contains a "uv-trampoline repair": if it is launched with a foreign interpreter (for example python_embedded_vc, Python 3.11.9), it purges foreign site-packages from sys.path and moves _venv_qwen_tts_457\Lib\site-packages to the front, so an ABI-compatible 3.11 pulls in this venv's torch 2.7.1 instead of its own torch 2.11.

3. What is downloaded and from where (with links)

3.1 Base models (HuggingFace)

3.2 Python and environment tools

3.3 Python packages (PyPI)

3.4 Rebuilding the environment from scratch (if the venv is lost)

REM uv downloads the required CPython 3.11 itself
uv venv _venv_qwen_tts_457 --python 3.11.14

REM CUDA build of torch (separate index)
uv pip install --python _venv_qwen_tts_457\Scripts\python.exe ^
  torch==2.7.1 --index-url https://download.pytorch.org/whl/cu128

REM the rest
uv pip install --python _venv_qwen_tts_457\Scripts\python.exe ^
  transformers==4.57.3 peft==0.19.1 accelerate==1.12.0 ^
  qwen-tts==0.1.1 soundfile==0.13.1 librosa==0.11.0 ^
  numpy==2.4.6 safetensors==0.7.0 huggingface-hub==0.36.2

Alternative for weights: huggingface-cli download Qwen/Qwen3-TTS-12Hz-1.7B-Base (the CLI lives in _venv_qwen_tts_457\Scripts\huggingface-cli.exe, or use python -m huggingface_hub.commands.huggingface_cli).

4. Step-by-step launch

Step 0. Check hardware and driver

nvidia-smi
python -c "import torch; print(torch.__version__, torch.cuda.is_available())"

Expected: 2.7.1+cu128 True. On a laptop with an RTX 3060 6 GB, the worker automatically installs a "display-headroom guard" (see section 6).

Step 1. Environment variables

cd /d B:\Downloads\Portable\NOESIS-VC-ONE\models\llm\NOESIS-3.5B-A0.5B-DUBBING-FILM\NOESIS-Qwen3-Omni-MoE-BF16
set PYTHONUTF8=1
set PYTHONIOENCODING=utf-8
set TRANSFORMERS_VERBOSITY=error

Step 2. Start the resident worker

The worker keeps the model in VRAM between requests — this is the main optimization (load ~10-30 s, then each clone 3-8 s).

REM Option A (recommended): the venv's own python
_venv_qwen_tts_457\Scripts\python.exe _venv_qwen_tts_457\_qwen_tts_worker.py

REM Option B (fallback): if the native python is broken, use embedded 3.11.9;
REM the worker repairs sys.path against the venv site-packages itself
B:\Downloads\Portable\NOESIS-VC-ONE\python_embedded_vc\python.exe _venv_qwen_tts_457\_qwen_tts_worker.py

The worker prints diagnostics to stderr and JSON replies to stdout (keeping them separate is critical: do not mix them).

Step 3. JSON-IPC protocol

The worker reads one JSON line from stdin and replies with one JSON line on stdout.

`clone` command — a single utterance:

{
  "cmd": "clone",
  "bundle": "NOESIS-Qwen3-TTS-1.7B-Talker-Combo-LLM2-ASR2-BF16",
  "ref": "C:\\ref\\speaker.wav",
  "ref_text": "exact text spoken in the reference",
  "text": "The text to synthesize.",
  "lang": "english",
  "out": "C:\\out\\take.wav",
  "adapter": "adapters\\NOESIS-Talker-Combo-germanic-LoRA-r8-BF16",
  "adapter_scale": 0.32,
  "x_vec_only": false,
  "max_new_tokens": 220,
  "temperature": 0.6,
  "top_p": 0.9,
  "repetition_penalty": 1.3
}

`clone_batch` command — a batch of utterances from one speaker (~2-3x faster):

{
  "cmd": "clone_batch",
  "bundle": "NOESIS-Qwen3-TTS-1.7B-Talker-Combo-LLM2-ASR2-BF16",
  "ref": "C:\\ref\\speaker.wav",
  "ref_text": "reference transcript",
  "lang": "english",
  "adapter": "adapters\\NOESIS-Talker-Combo-germanic-LoRA-r8-BF16",
  "adapter_scale": 0.32,
  "top_p": 0.9,
  "repetition_penalty": 1.3,
  "items": [
    {"text": "First line.", "out": "C:\\out\\a.wav"},
    {"text": "Second line.", "out": "C:\\out\\b.wav", "temperature": 0.7}
  ]
}

`shutdown` command — unload the model and exit:

{"cmd": "shutdown"}

`clone` reply (ok):

{"ok": true, "dur_s": 3.42, "sr": 24000, "gen_s": 4.8,
 "out": "C:\\out\\take.wav", "load_s": 12.4, "vram_gb": 4.61}

Error reply:

{"ok": false, "err": "ValueError('ref_text required when x_vector_only_mode=False')", "trace": "..."}

Step 4. Example Python driver (full cycle)

Save as run_talker.py next to the bundle and run it with any Python 3.11 (for example python_embedded_vc):

import json, subprocess, sys, os
from pathlib import Path

OMNI = Path(r"B:\Downloads\Portable\NOESIS-VC-ONE\models\llm\NOESIS-3.5B-A0.5B-DUBBING-FILM\NOESIS-Qwen3-Omni-MoE-BF16")
VENV_PY = OMNI / "_venv_qwen_tts_457" / "Scripts" / "python.exe"
WORKER  = OMNI / "_venv_qwen_tts_457" / "_qwen_tts_worker.py"
BUNDLE  = OMNI / "NOESIS-Qwen3-TTS-1.7B-Talker-Combo-LLM2-ASR2-BF16"

env = dict(os.environ, PYTHONUTF8="1", PYTHONIOENCODING="utf-8")

p = subprocess.Popen(
    [str(VENV_PY), str(WORKER)],
    stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE,
    text=True, encoding="utf-8", env=env, bufsize=1,
)

def call(req: dict) -> dict:
    p.stdin.write(json.dumps(req, ensure_ascii=False) + "\n")
    p.stdin.flush()
    return json.loads(p.stdout.readline())

take = call({
    "cmd": "clone",
    "bundle": str(BUNDLE),
    "ref": r"C:\ref\speaker.wav",
    "ref_text": "exact reference transcript",
    "text": "A speech synthesis test in English.",
    "lang": "english",
    "out": r"C:\out\en_take.wav",
    "adapter": "adapters\\NOESIS-Talker-Combo-germanic-LoRA-r8-BF16",
    "adapter_scale": 0.32,
})
print("RESULT:", take)

call({"cmd": "shutdown"})
p.wait()

5. Adapters (LoRA) — how to apply them

adapters/ contains 15 family LoRAs (BF16). Rule: exactly one adapter at a time, adapter_scale = 0.32. Never sum adapters into one checkpoint — that destroys EOS/codec (rule R-TALKER-112LANG-PER-FAMILY-SWAP-NOT-MERGE).

The worker does PeftModel.from_pretrained -> sets scaling = 0.32 -> merge_and_unload(). On a family change the base is reloaded (~16 s), because a merge cannot be undone.

The 10 native Qwen3-TTS languages (en, ru, zh, de, fr, es, ja, ko, it, pt) normally run without an adapter (plain Combo). ar in this bundle goes through semitic.

LoRA configuration example (one family): r=8, lora_alpha=16, dropout=0.05, target_modules = [q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj], task_type = CAUSAL_LM.

6. Worker environment variables

7. Troubleshooting

8. Hardware and performance

9. Loading flow (what happens under the hood)

Qwen3TTSModel.from_pretrained(bundle, device_map="cuda:0", dtype=torch.bfloat16)
        │
        ├── talker/ (28 layers, hidden=2048)   <- 12 Hz codec-frame generation
        ├── code_predictor/ (5 layers)          <- 16 code groups
        └── speech_tokenizer/ (Qwen3TTSTokenizerV2, 12 Hz) <- codec -> wav 24 kHz

adapter:  PeftModel.from_pretrained(inner) -> scaling=0.32 -> merge_and_unload()
call:     model.generate_voice_clone(text, language, ref_audio, ref_text, ...)
output:   24 kHz wav (numpy float32)

Key IDs from config.json: tts_bos=151672, tts_eos=151673, tts_pad=151671, codec_bos=2149, codec_eos=2150, codec_think=2154, codec_nothink=2155. Rate: position_id_per_seconds = 13, sample_rate = 24000 Hz.

10. Comparisons and benchmarks

All six comparisons use the same nine engines: NOESIS Talker Combo (ours), Qwen3-TTS, Higgs Audio v3, CosyVoice 3.0, Fish Audio S2, Chatterbox Multilingual v3, IndexTTS2, XTTS v2, Seed-VC.

10.1 Languages supported

Our 112 is a measured NOESIS route test (112/112); competitor counts are the numbers their own model cards publish.

10.2 Voice quality (operator blind listen-off)

Ranked best to worst by blind listening on clean reference audio. Our engine is best. Benchmarked on ours with Seed-TTS Eval (WER / SIM), UTMOSv2, SECS / ReDimNet and PESQ.

Objective quality on our engine: UTMOSv2 2.04 to 2.3 (82% of studio ground truth 2.82) and PESQ 3.2 to 3.84. Reference scale (WideCodec, UTMOSv2 ~1 to 5): ground truth 2.82, nemo44k 2.90, WideCodec 2.79, DAC 2.67, SNAC 2.34, EnCodec 2.04. Best-of-N selection lifts the pipeline from about 1.1 to about 2.3.

10.3 Model size (parameters)

10.4 Licence (open-source friendliness)

10.5 Download size on HuggingFace (GB, approximate weight size)

10.6 VRAM to run (GB)

Ours and Qwen3-TTS share the 1.7B base and both fit a 6 GB laptop GPU.

10.7 Honesty rule

Language coverage is our measured 112/112 NOESIS route test; competitor language counts come from their published cards. Voice-quality ranks are the operator blind listen-off; objective numbers are our measured runs. Model sizes, download sizes, VRAM and licences are approximate / per publisher. A missing value is shown as n/a. No fabricated numbers. Check each competitor card before quoting.

Base component references