laion/moss-voice-profile-loras-500

MOSS voice-profile LoRAs — 500 voices, rank 4

<!-- moss-tied-heads-warning -->

⚠️ Do not merge this adapter into the base weights

merge_and_unload(), merge_adapter(), and any offline "bake the LoRA into the checkpoint" script will destroy the model irrecoverably. This is not a performance caveat. Read this before you write a deployment script.

Why

This adapter targets audio_lm_heads.0 … audio_lm_heads.11 and text_lm_head — 12 of its 23 target modules. In this architecture those output heads are weight-tied to the input embeddings: tie_weights() sets

audio_lm_heads[i].weight  IS  audio_embeddings[i].weight     # the same tensor, not a copy
text_lm_head.weight       IS  transformer.embed_tokens.weight

They are one allocation with two names. So when a merge adds B @ A * (alpha/r) into the head weight, it writes that delta straight into the embedding table at the same time. The model then reads its own inputs through a matrix that has been shifted by an output-side correction. Generation does not fail loudly — it degrades into noise or into a fixed babble, and the damage is inside the checkpoint you just saved. There is nothing to unmerge afterwards, because the original values are gone.

Verify it yourself in three lines

Do not take our word for it:

m = base.model if hasattr(base, "model") else base
print(m.audio_lm_heads[0].weight.data_ptr() == m.audio_embeddings[0].weight.data_ptr())
# True  -> same storage, merging corrupts the embeddings

What to do instead

Load with PEFT and leave the adapter unmerged. Set its strength through the scaling factor:

from peft import PeftModel

model = PeftModel.from_pretrained(base, "<this repo>", adapter_name="a").to(dev).eval()
# do NOT call model.merge_and_unload()

def set_weight(model, name, w):
    """Scale one named adapter's contribution.  alpha/r is its own base scaling."""
    for module in model.modules():
        scaling = getattr(module, "scaling", None)
        if isinstance(scaling, dict) and name in scaling:
            if not hasattr(module, "_base_scaling"):
                module._base_scaling = {}
            module._base_scaling.setdefault(name, scaling[name])
            scaling[name] = module._base_scaling[name] * float(w)

set_weight(model, "a", 1.0)
model.base_model.set_adapter(["a"])      # several adapters can be active at once

This sounds identical to a merge. An unmerged LoRA computes Wx + (B @ A)x * (alpha/r), which is exactly what the merged weight W + B @ A * (alpha/r) would compute — the same arithmetic, in a different order. You give up a small amount of inference speed and you keep the ability to change the weight, stack several adapters, or turn one off. Nothing about the sound changes.

If you are stacking adapters

Set each one's scaling separately and activate them together with model.base_model.set_adapter([...]). Note that stacking is not free: in our own measurements a deep stack held audio quality but destroyed intelligibility (word error 0.063 → 0.554). Add adapters deliberately and measure.

If you maintain code that merges

A regex over module names is not enough — the reliable test is identity of storage. Group the modules by weight.data_ptr() and refuse to merge into any group with more than one member. lora_bank.py in LAION-AI/Humaneness-Voice-Demo-Server does this and asserts on the merge path.

500 low-rank adapters for [`laion/moss-tts-local-transformer-4.55b-voice-acting-v2`](https://huggingface.co/laion/moss-tts-local-transformer-4.55b-voice-acting-v2), one per synthetic voice profile. Each adapter makes the frozen base model speak as one specific, invented character voice — a "grave documentary narrator", an "anxious teenage hacker" — and ships with the reference clip and the profile card that define that character. Every one of them is rank 4, alpha 8, dropout 0.05 over the same 23 target modules, weighs about 34 MB, and beats the frozen base model on that voice's own held-out clips. None of them is a clone of a real person: every voice profile is synthetic, invented from a design spec and then generated.

If you have never seen this project before, read §1 to §3 and then run quickstart.py.

• —500 voices · 17.36 GB · ~34.4 MB per adapter · 2,506 files
• —8,589,312 trainable parameters per adapter — 96.1 % of them in the semantic backbone, only ~3.9 % in the acoustic path (§4)
• —Median gain over base: 0.1571 nats of held-out loss (min 0.0751, max 0.3150). All 500 beat the base; none was shipped that did not.
• —Base model `laion/moss-tts-local-transformer-4.55b-voice-acting-v2` · codec `OpenMOSS-Team/MOSS-Audio-Tokenizer-v2` · reference audio voices/<voice>/reference.wav, in this repo (why not the dataset that sounds like it)

Read this first

Not buried at the bottom, because these change how you should use the release. Full detail in §12.

1. 1.No human has listened to any of this in a controlled study. Every quality number here is held-out language-modelling loss on MOSS audio codes, plus ECAPA speaker cosine. Those are proxies for "sounds like the voice", not measurements of it.
2. 2.You need the reference clip as well as the weights. The base model is reference-conditioned. An adapter with no reference gives you a random speaker wearing the adapter's colouring — there is no useful "no reference" mode. The right clip ships at voices/<voice>/reference.wav.
3. 3.8 of the 500 adapters are below the measured saturation point and 6 of those are drawn from voices the generator never cloned reliably. They are flagged and named (§8), not hidden.
4. 4.Do not read a voice's gender out of `caption_general`. The corpus renders the gender and background-noise axes of that prose field with inverted polarity, and about 35 % of training instructions carried the buggy prose verbatim. Use `card_gender` from profile.json / the manifest (§12).

Relationship to TTS-AGI/moss-voice-profile-loras

There is an earlier, smaller LoRA repo, and it is not superseded silently, so here is what it actually is.

`TTS-AGI/moss-voice-profile-loras` (public, 168 files) is the 10-voice pilot and rank-ablation study that this release was designed from. It holds ten voices, each with a shipped adapter and six ablation arms (ranks/{r4,r8,r16}/{stage1,stage2}) plus ablation/ result tables. That sweep is where rank 4 was chosen; it is why every adapter here is rank 4 and not rank 16.

All ten of its voices — anime_088, emolia_c0542, emolia_c1682, emolia_c1699, emolia_c2570, k10_age3_bg1, k325_age3_bg1, k395_age3_bg1, k91_age5_bg0, mediathek_0184 — are present in this release, and the pilot repo's own README says so, describing this build as the one whose voices "will replace these same ten".

Keep the pilot repo if you want the rank ablation or the two-stage arms. For actually generating a voice, use this one.

1. What these are

A voice profile here is a stable, invented speaker identity — not a real person and not a recording of one. Each profile started as a written design spec (name, tagline, gender, age, accent, register, timbre sliders, casting notes) and a single reference clip. That profile was then used to generate a large corpus of takes across emotion, delivery, character and edge-case prompts — roughly 38,000 candidate takes per voice. The adapter shipped here is fitted on the subset of those takes that most convincingly sound like the profile, as scored by an ECAPA speaker embedder against the reference.

The result is a 34 MB file that, attached to the 4.55 B base model, holds that one identity across the whole expressive range the corpus covers: emotions, deliveries, vocal bursts, English and German. Nothing in this release is trained on a real speaker's voice, and there is no real identity to recover from these weights.

These are stage-1 identity adapters (stage = "stage1" on all 500 rows). They are complete and usable as they are.

2. What is in the repo

README.md                  this file
MANIFEST_COLUMNS.md        the column dictionary, also inlined at §10
manifest.parquet           one row per voice, 500 rows x 56 columns
manifest.json              the same rows as JSON
RELEASE.json               build summary + the downstream consumer contract
quickstart.py              runnable end-to-end example
voices/<voice>/
    adapter_model.safetensors    the LoRA weights, peft format  (~34.4 MB)
    adapter_config.json          the peft config
    reference.wav                the voice's reference clip -- REQUIRED at generation time
    profile.json                 the voice's profile card (name, tagline, casting, tags)
    voice.json                   this voice's manifest row, standalone

500 voice directories × 5 files + 6 top-level files = 2,506 files, 17,359,449,359 bytes.

Choosing a voice. Use the profile-card columns in the manifest — voice_name, tagline, card_gender, card_age, card_accent, card_register, card_tags. These come from the design spec each voice was generated from. Do not use the corpus's caption_general prose for this (§12).

import pandas as pd
m = pd.read_parquet("manifest.parquet")
m[m["card_tags"].str.contains("narrator", na=False) & (m["card_gender"] == "Female")] \
 [["voice", "voice_name", "tagline", "gain"]].sort_values("gain", ascending=False).head()

pandas gotcha: rank is also a DataFrame method. Write m["rank"], never m.rank. The same applies to any column whose name collides with a method.

If you are consuming this release from code, read manifest.parquet and treat voice as the key. Every row already points at that voice's final adapter — the choice between a retrain and its incumbent has already been made, so there is nothing to resolve:

m = pd.read_parquet("manifest.parquet").set_index("voice")
rel  = m.loc["emolia_c0019", "adapter_dir"]        # "voices/emolia_c0019" -- relative, use this
was_new = bool(m.loc["emolia_c0019", "retrained"]) # did this release retrain it?
why  = m.loc["emolia_c0019", "retrain_status"]     # shipped_retrain / rejected_worse / ...

The *_abs path columns and release_root record absolute paths on the machine that built the release and are provenance only; on the Hub, use adapter_dir. Never load from adapter_src — it points into build trees that are not part of this release.

3. How to use one

quickstart.py in this repo is the code below, runnable:

python quickstart.py --release . --voice emolia_c0019 --out hello.wav

Budget roughly 20 GB of VRAM for the base model plus the codec.

Three things actually trip people up:

(a) You need the reference clip as well as the adapter. Covered above; quickstart.py defaults to the shipped one, so in practice you do not have to find it.

(b) `PeftModel.active_adapter` is a plain instance attribute, not a property. It is assigned once in PeftModel.__init__ and is not kept in sync by everything that changes adapters — on the class itself it is simply None. `active_adapters` (plural) IS a property and does query the model. Read the plural one, or read peft_config directly, and never assume the singular attribute is current after juggling adapters.

(c) peft 0.20.0's offline path mangles `subfolder`. With HF_HUB_OFFLINE=1, passing subfolder= to PeftModel.from_pretrained puts the subfolder into the filename and passes it again as a kwarg, so it looks for <voice>/<voice>/adapter_model.safetensors and raises LocalEntryNotFoundError. Resolve to a plain local directory first, as below, and it works either way.

import numpy as np, soundfile as sf, torch
from huggingface_hub import snapshot_download
from peft import PeftModel
from transformers import AutoModel, AutoProcessor

BASE  = "laion/moss-tts-local-transformer-4.55b-voice-acting-v2"
CODEC = "OpenMOSS-Team/MOSS-Audio-Tokenizer-v2"
REPO  = "laion/moss-voice-profile-loras-500"
VOICE = "emolia_c0019"                      # any directory name under voices/

# Pull just the one voice (~34 MB + a reference clip), not all 17 GB.
root = snapshot_download(REPO, allow_patterns=[f"voices/{VOICE}/*", "manifest.parquet"])
adir = f"{root}/voices/{VOICE}"

proc = AutoProcessor.from_pretrained(BASE, trust_remote_code=True, codec_path=CODEC)
proc.audio_tokenizer = proc.audio_tokenizer.to("cuda").eval()

# "some weights of MossTTSLocalModel were not initialised" warnings about audio_lm_heads /
# text_lm_head are benign -- those tensors are weight-tied to the embeddings, not missing.
model = AutoModel.from_pretrained(BASE, trust_remote_code=True,
                                  dtype=torch.bfloat16, attn_implementation="sdpa").cuda().eval()

model = PeftModel.from_pretrained(model, adir).eval()      # a plain local directory
name  = model.active_adapters[0]                           # PLURAL: the property (see (b))
print("rank:", model.peft_config[name].r, "alpha:", model.peft_config[name].lora_alpha)

conv = [[proc.build_user_message(
    text="I have read the file, and that is exactly what worries me.",
    instruction="A warm, unhurried voice, speaking just above a murmur.",
    language="English",                       # "English" or "German"
    reference=[f"{adir}/reference.wav"],      # ships with the adapter -- do not omit
    tokens=12)]]                              # a length HINT in words, not a limit
batch = proc(conv, mode="generation")

out = model.generate(input_ids=batch["input_ids"].cuda(),
                     attention_mask=batch["attention_mask"].cuda(),
                     max_new_frames=400,      # the real ceiling: 400 frames ~ 32 s at 12.5 fps
                     do_sample=True,
                     text_temperature=0.7, text_top_k=50, text_top_p=1.0,
                     audio_temperature=1.0, audio_top_p=0.95, audio_top_k=30,
                     audio_repetition_penalty=1.1)

msg = proc.decode(out)[0]
if not msg.audio_codes_list:      # a NORMAL, silent failure mode of this model -- always check
    raise SystemExit("empty decode; retry with another seed")
w = msg.audio_codes_list[0].cpu().float().numpy()
sf.write("out.wav", np.ascontiguousarray(w.mean(0) if w.ndim > 1 else w), 48000)

To merge an adapter permanently: model = model.merge_and_unload(). These adapters were evaluated at scale 1.0 only; no other merge scale was measured, and nothing here was measured with two voice adapters attached at once.

4. What the adapter actually touches

Read straight out of the shipped adapter_model.safetensors, not out of the config. 536 tensors, 268 adapted modules, 8,589,312 trainable parameters, identical in shape across all 500 voices.

Where those 8.59 M parameters go:

This is the single most useful fact for deciding how to use these adapters. About 96 % of the capacity sits in the semantic/prosodic transformer and only about 3.9 % in the acoustic path — and of the local transformer's stack, only layer 0 is adapted at all. These adapters overwhelmingly change how the model decides what the voice does — phrasing, timing, prosodic shape, register — and only marginally retune the codebook decoding on the way out. If you expected a timbre patch, this is not one; the timbre still comes substantially from the reference clip, which is why §3(a) matters as much as it does.

All 500 adapters share this exact structure: rank = 4, lora_alpha = 8 and n_target_modules = 23 on every manifest row, and all 500 adapter_sha256 values are distinct.

5. How they were trained

Per voice:

1. 1.Take the voice's ~38,000 generated candidate takes and drop the empty ones, anything under 0.5 s, and every clip belonging to a held-out group (40 groups per voice, stratified by block and language, fixed seed). Held-out audio never enters training.
2. 2.Select the training pool by ECAPA speaker similarity against the voice's reference (§6, §7).
3. 3.Encode the pool to MOSS audio codes — 12 codebooks × 1024, 12.5 frames/s exactly (hop 3,840 at 48 kHz).
4. 4.Train rank 4 / alpha 8 / dropout 0.05 on the 23 target modules for 6 epochs, checkpointing and evaluating on the held-out clips after every epoch.
5. 5.Ship the epoch with the lowest held-out loss. If no epoch beat the frozen base model, ship nothing. That never happened: all 500 beat their base.

Captions are resampled every epoch from the stored measurements, so a clip is not welded to one phrasing of its description.

Why best-epoch selection rather than a fixed epoch count. Across the 499 runs of the original build the winning epoch was ep2 ×90, ep3 ×297, ep4 ×99, ep5 ×13 — there is no constant that is right for all voices. Every one of those runs overfits: the best epoch was never the last (499/499). But the rise after the minimum is strictly monotone in only 472/499 (94.6 %), so "it gets worse after the best epoch" is reliable while "it gets worse at every subsequent epoch" is not — an early-stop on the first uptick would misfire on about 5 % of voices. Picking the minimum over all six is what makes the 100 % beat-base rate real rather than lucky.

Across the shipped 500 adapters (which for 119 voices means the retrain's epoch, not the original run's) the chosen epoch is ep2 ×54, ep3 ×331, ep4 ×101, ep5 ×13, plus stage2 ×1 for emolia_c1699.

Cost. Summing gpu_h over the shipped adapters gives 378.7 GPU-hours (repair build 329.3, 3,000-clip retrain 47.3, floor-capped retrain 0.8, reference study 1.2). That column records only the stage that produced the shipped weights; counting the discarded challengers too, the build reports roughly 355 GPU-h for the original 499 plus 51 GPU-h for the retrain pass, ≈406 GPU-h in total. Encoding time is not included in either figure.

Provenance of the 500. 499 come from the identity-repair pipeline (src_tree = repair, lora3k or lora3k_fc); the 500th, emolia_c1699, is the original single-voice reference study and used a different, two-stage recipe (§8).

6. The two thresholds, and what they do and do not mean

Two numbers govern which clips a voice was trained on. They are both cosine similarities between ECAPA speaker embeddings, on a scale internal to this project, and they are easy to confuse.

The original recipe took every take at spk_sim ≥ 0.60, capped at the 6,000 highest. If fewer than 800 cleared, it relaxed to that voice's own top 800 and flagged the voice relaxed; below 200 it refused to train at all. The retrain recipe replaced this with the voice's top 3,000 takes with a hard 0.40 floor (§7).

These are conventions of this project, not established perceptual facts. The project's own report is explicit: "The 0.40 floor is this project's ECAPA scale, not an established perceptual fact: an independent WavLM-tbr embedder puts 75.4 % of the selected failures above its own same-speaker threshold (0.472)." Two embedders disagree about three quarters of the rejected population.

Three different speaker embedders appear around this project and none of their scales are interchangeable. Worth spelling out, because a threshold copied across them is silently wrong:

The 0.80 in particular belongs to -id and must not be applied to -tbr or to ECAPA. A cosine is only meaningful in the embedding it was computed in.

`min_spk_effective` in the manifest is the number that actually matters per voice: the lowest similarity that really entered that voice's pool, whatever rule produced it. Across the release: 378 voices realise ≥ 0.60, 119 land in 0.40, 0.60), **2 fall below 0.40** (`emolia_c1340` at 0.388, `emolia_c2142` at 0.387 — both kept pre-floor incumbents, see [§8), and 1 is null (emolia_c1699, whose pool was not similarity-filtered).

One more caveat from the same report, about what hard identity filtering costs: "Identity is bought, not free: the reference run paid −11 % target strength and −14 % blend for its identity gain." Selecting for clips that sound like the profile is not the same as selecting for clips that are expressive or well articulated.

7. Why 3,000 clips

Over the 499 adapters of the original build, gain over base correlates with pool size at r = +0.532 and with pool purity (mean speaker similarity) at only r = +0.234. And it saturates. Measured on the pre-retrain state of all 500 voices:

3,000–4,999 and 5,000–6,000 are indistinguishable. Going past ~3,000 clips buys nothing; staying under it costs a lot. The 123 voices trained on fewer than 3,000 clips had a median gain of 0.1221 against 0.1627 for the rest — about 25 % less improvement.

So those 123 voices, and only those, were retrained on the voice's top 3,000 takes by speaker similarity with a hard 0.40 floor — not on a flat 0.60 cut, because 124 of the 500 voices cannot reach even 800 clips at 0.60, which is exactly why they were under-trained in the first place. The other 377 were left alone: they are at or above saturation, and retraining them would have burned GPU-hours to move a number that has stopped moving.

Never ship a worse adapter. Every challenger was written to a separate tree and evaluated on the same held-out clips as the incumbent — the validation container was copied verbatim, not rebuilt, which is the only way two losses are comparable. The lower loss wins.

119 voices shipped a new adapter, 4 kept their incumbent, and 0 shipped worse — that last figure guaranteed by construction, since a losing challenger is discarded. Median gain across those 123: 0.1221 → 0.1427, a median per-voice improvement of +0.0219 (median +0.0227 among the winners). Voices below saturation: 123 → 8.

Pool composition beats pool size

Of the 123 retrain targets, 117 could fill 3,000 clips from their own original takes above the floor; 6 could not and had their pools topped up with audio regenerated by the identity-repair pipeline.

Every pool containing regenerated audio regressed, without exception. All-original pools overwhelmingly improved, but not universally: emolia_c2305 and k492_age2_bg1 had clean 3,000-clip retrains that still lost to their incumbents, and shipped the incumbent. No shipped adapter in this release contains a single regenerated clip — pool_n_repaired = 0 and used_repaired = false on all 500 rows.

Gain by pool size across the shipped release now looks like this:

8. Which voices are weak, and why

Honesty matters more here than a tidy release. 8 of the 500 adapters are still below the 3,000-clip saturation point and should be expected to clone their profile less reliably than the rest. They are flagged below_saturation = true in the manifest and named here:

The six sub-floor voices

The most informative result in the release. Six voices could not supply 3,000 original takes above the 0.40 identity floor — their generator failed to sound like the profile most of the time. Their incumbent adapters had been trained on 800 clips selected by dropping the threshold until 800 existed, which for emolia_c1340 and emolia_c2142 meant reaching down to similarity 0.388 and 0.387, i.e. below the identity floor, including clips that are arguably a different speaker. Two repairs were tried, and they came out opposite ways.

Attempt 1 — more clips, from regenerated audio. Failed, six times out of six. Topping the pool up with identity-repair regenerations reached a full 3,000 clips at a far higher effective similarity threshold for every one of them. Every one came out worse than the 800-clip incumbent, losing roughly a third to a half of its gain. The regenerated audio is measurably degraded on content and delivery — for emolia_c1070, word error rate +96 %, naturalness −23 %, burst blend −38 % — and the validation clips are original audio. An adapter fitted on regenerated audio predicts regenerated audio.

Attempt 2 — fewer clips, but all original and all above the floor. Won, up to a point.

What this shows. The saturation curve in §7 is not about clip count as such — it is about the count of clips drawn from the distribution you are evaluated on. Three thousand regenerated clips lose to 800 originals. But cleanliness stops paying once the pool gets small enough: emolia_c1340 (546 clean clips) and emolia_c2142 (632) both lost to their 800-clip incumbents even though those incumbents contain sub-floor audio, while emolia_c1070 won on 1,096. The crossover therefore sits between 632 and 1,096 clips: above it a smaller clean pool wins, below it there is simply not enough data and the larger contaminated pool is the better bet. That is considerably more useful than "3,000 is the magic number", and it is the opposite of what a naive reading of the saturation table predicts.

The two voices whose pools reach below 0.40 are also the two with the lowest gains in the entire release — at least consistent with treating the floor as a training constraint and not only a reporting one.

All six remain the weakest voices here and stay flagged sub_floor_voice = true. Improved or not, each is fitted on a pool far under saturation, drawn from a voice the generator never cloned reliably. Expect less consistent identity from all six, and prefer other voices where you have the choice.

emolia_c1699 is not like the others

emolia_c1699 is the original single-voice reference study the whole project was built from. Its adapter comes from a different recipe — a two-stage curriculum over 18,823 unfiltered clips (plus a 1,580-clip stage-2 sharpening pass), selected by a rank sweep (4 / 8 / 16) and a stage sweep rather than the single-stage similarity-filtered recipe used for the other 499. It is rank 4 like the rest and its held-out loss is measured on the same kind of split, but it is not a comparable data point about the recipe — it has the highest gain in the release (0.2253) partly because it is the only voice trained this way. recipe = reference_study_2stage marks it, and pool_mean_spk / min_spk_effective are null for it because its pool was never similarity-filtered.

9. profile.json, reference.wav, and which gender field to trust

`reference.wav` is the voice's conditioning clip — the same clip the profile was defined by and the corpus was generated from, at the enhancement variant the profile card rates best by DNSMOS (reference_variant in the manifest says which of orig / sidon / cbx; across the release: 173 / 156 / 171). The base model is reference-conditioned, so this file is not optional. These 500 files are the only published copy of the profiles' reference clips — see the note below.

`profile.json` is the voice's design spec — the document the voice was generated from, not a measurement of the audio that came out:

{
  "cid": "emolia_c0019",
  "name": "Grave Documentary Narrator",
  "tagline": "A weightful and measured voice that conveys historical gravity and calm authority.",
  "gender": "Male",
  "age": "Late 40s to 50s",
  "language": "English",
  "accent": "Standard American",
  "register": "Mid-range baritone with a calm, downward inflection",
  "timbre_profile": {"metallic": 1, "throat_guttural": 2, "falsetto": 0,
                     "chest_voice": 4, "roughness": 2, "brightness": 3},
  "distinctive_features": "Precise enunciation combined with a somber, grounded resonance…",
  "emotional_range": "Primarily serious and informative with a steady, cautionary undertone.",
  "casting": {"classic_fantasy": {"role": "Royal Archivist", "direction": "…"},
              "sci_fi": {…}, "mystery_horror": {…}, "contemporary": {…}},
  "tags": ["english", "standard american", "male", "middle-aged", "narrator", …]
}

The casting blocks are genre-specific role + direction pairs and make good ready-made instruction= strings.

`card_gender` is the field to trust. It is the profile card's gender, i.e. the intended design, and it is not derived from the corpus's buggy caption_general prose (§12). Anyone wanting a voice's gender should read card_gender (or profile.json's gender) and nothing else. Note it states intent, not a measurement of the produced audio. Across the release: Male 242, Female 236, Androgynous 21, null 1.

One voice, `k532_age3_bg1`, has a stub profile card with no descriptive fields, so voice_name, tagline and the card_* columns are null there. Its adapter and reference clip are unaffected. card_language is additionally null for 115 voices whose cards did not record one.

9b. The speaker name — name, and a third gender field

Every voice now also carries a given name: one unique, ordinary first name per profile, added so that a model can be conditioned on "speak as Katrin" instead of on a reference clip. It is in manifest.parquet / manifest.json (name, name_gender_class, name_confidence), in every voices/<voice>/voice.json, and standalone in the new top-level `names.csv` / `names.json`. profile.json is deliberately left byte-untouched.

"name": "Katrin", "name_gender_class": "female", "name_confidence": 0.86

Names are decorative labels for conditioning. They are not the character names — those stay where they were, in voice_name (manifest, voice.json) and in name (profile.json, unchanged).

`name_gender_class` is measured, and it is not the same thing as `card_gender`.

A voice is male when at least 85 % of its takes score vn_GEND_reg >= 3.2103 (the 50/50 boundary of a 2-component Gaussian mixture fitted to 1,003,974 clips: modes at 1.587 and 4.867), female when at most 15 % do, and uncertain in between — 209 male, 170 female, 121 uncertain. High vn_GEND_reg is masculine (it correlates +0.87 with chest resonance and −0.55 with head resonance across the 500 per-voice means); the inverted prose in caption_general in the older corpus release is a captioning bug and was never the source here.

The two labels agree on 89.2 % of the 379 voices this classification calls confidently — which is also a fourth independent confirmation of the polarity, since an inverted ladder would give ~11 % — and they disagree on 41. names.csv flags every one (agrees_with_card = NO).

The 41 track speaker-identity failure. Disagreement rate falls monotonically with how well a voice reproduces its own reference: 15.8 % in the weakest quartile by frac_ge06 (share of takes with spk_sim >= 0.60) down to 3.2 % in the strongest. frac_ge06 differs significantly between agreeing and disagreeing voices (0.248 vs 0.181, Welch p = 0.011), as does min_spk_effective (p = 0.032). By family: char 21.1 %, emolia 10.4 %, mediathek 7.0 %, anime and refvoice 0 %.

It does not explain every case, though: the bare weak/strong split is not significant on its own (chi-square p = 0.10), and seven of the ten largest disagreements are char voices specced Androgynous that measure 5.1–5.3 — vague spec rather than failed generation. §12's caveats apply; nobody has listened to any of them.

The `uncertain` bucket is mostly measured, not designed, ambiguity. Of the 121, only 12 are specced Androgynous; 69 are specced Female and 40 Male but do not render consistently on one side. Within-voice std(vn_GEND_reg) averages 1.288 there against 0.759 for decided voices — they genuinely swing take to take. Filter card_gender == "Androgynous" if you want only the designed-androgynous voices.

Reference audio is here, not in moss-voice-profile-references

Measured 2026-08-23 from that repo's own metadata.parquet, because earlier versions of this card described it as the reference set for these voices and that is wrong:

It is a single-voice reference / voice-conversion ablation study. Useful for what it is, but it does not contain the reference clips of these 500 voices and never did.

The 500 reference clips are the `voices/<voice>/reference.wav` files in this repo, one per profile, with reference_file / reference_variant / reference_best_version in manifest.parquet. There is no standalone published dataset of them, so this release is the reference-audio artefact.

10. manifest.parquet — the 56 columns

One row per voice, 500 rows, 56 columns. manifest.json carries identical rows, and each voice's own row is duplicated standalone at voices/<voice>/voice.json.

Identity and recipe

Training pool

Training and evaluation

Retrain bookkeeping

Quality flags

Reference clip

Profile card

Paths and integrity

Useful slices:

import pandas as pd
m = pd.read_parquet("manifest.parquet")

m.nlargest(10, "gain")[["voice", "voice_name", "pool_rows", "gain"]]     # the strongest
m[m["below_saturation"] | m["sub_floor_voice"]]                          # everything flagged weak
m[m["retrain_attempted"]].groupby("retrain_status")["gain"].describe()   # what the retrain did

11. Related repositories

A further consolidated reference-voice dataset exists internally (TTS-AGI/moss-reference-voices-consolidated) but is private and not linkable — it is named only so the provenance chain is complete.

12. Limitations in full

• —They are not validated perceptually. Every quality number in this release is a held-out language-modelling loss on MOSS codes, plus ECAPA speaker cosine. No listening test was run. "Higher gain" means "the model predicts this voice's audio tokens better", which is a proxy for, not a measurement of, how much it sounds like the voice.
• —`gain` is not strictly comparable across voices. Each voice has its own held-out set, so absolute losses live on different scales. The valid comparison is adapter vs. base within a voice — which is exactly what gain is.
• —8 voices are below the saturation point and 6 are sub-floor. They ship flagged and named (§8), not silently.
• —They are not speaker clones of real people. Every profile is synthetic. There is no real identity to recover from these weights. Please do not present output as a real person's voice.
• —They are not a substitute for the reference clip. Generation without a matching reference gives a random speaker with the adapter's colouring.
• —The 0.40 and 0.60 thresholds do not transfer to another speaker embedder (§6).
• —Identity is bought with expressiveness. Hard filtering on speaker similarity selects for clips that sound like the profile, not for clips that are expressive or well articulated; the project's reference run measured −11 % target emotion strength and −14 % blend for its identity gain.
• —They are not stackable. Nothing here was measured with two voice adapters attached at once, or at a merge scale other than 1.0.
• —Languages are English and German only. The corpus contains nothing else.
• —They inherit the base model's failure modes, including empty decodes — always check audio_codes_list before using the output.
• —Gender and background-noise prompting is unreliable. See below.

The caption_general polarity bug

Two of the descriptive axes in the corpus's rendered caption_general prose — GEND (gender) and BKGN (background noise) — are rendered with inverted polarity. It is a bug in the prose only: the numeric vn_GEND_* / vn_BKGN_* columns are correct and authoritative.

Why it affects these adapters, and how much. The training instruction for each clip is regenerated every epoch from the numeric VoiceNet buckets, and that path is unaffected. But with probability 0.35 the generator instead uses the clip's authored caption_general verbatim — the buggy prose. So roughly a third of the instructions each adapter saw during training carried inverted gender and background-noise wording. This applies to all 500 adapters equally, retrained and untouched alike. It is a property of the corpus these adapters were built from, and it cannot be corrected by retraining alone.

Practical consequence: do not rely on gender or background-noise wording in a prompt to steer these adapters on those two axes — the association they learned there is unreliable. All other axes are unaffected. And do not read a voice's gender out of `caption_general`; use `card_gender` (§9). This README deliberately never describes a voice using caption_general.

<details> <summary><b>How the polarity was established, and why the obvious test cannot work</b></summary>

caption_general is generated deterministically from the numeric buckets, so it agrees with them by construction whichever way the ladder points. Comparing the prose against the bucket therefore proves nothing — a keyword check of exactly that kind over 1,112 gender-mentioning captions across 47 voices returned 52 % agreement, i.e. chance. The polarity has to be pinned against acoustic correlates instead:

Chest-versus-head resonance is the physiologically correct masculine/feminine marker, and all four correlates agree: the bucket the prose calls "feminine" is acoustically the masculine end. For BKGN the same approach gives corr(vn_BKGN_reg, vn_RCQL_reg) = +0.786 and corr(vn_BKGN_reg, qual_background_quality) = +0.402, the latter from an independent model head. Caveat: this is an acoustic-correlate argument. No listening test was run.

</details>

Provenance note: the corpus's published MOSS codes are unusable

The adapters here were trained from MOSS codes re-encoded directly from the source mp3s. They do not use the *.moss.npy arrays published in the vprof_base / vprof_repaired WebDataset copies of the corpus, and neither should you.

Those published codes encode half-speed audio. The corpus tokeniser decoded mp3s through a path that did frame.to_ndarray().reshape(-1), which is correct only for mono; PyAV returns (channels, nb_samples) for planar layouts and (1, nb_samples × channels) for packed ones, and a blind flatten turns both into a signal of length nb_samples × channels. The voice-profile audio is duplicated mono written as a 2-channel mp3 (max |L−R| = 0.000000), so the flatten produced [s0,s0,s1,s1,…] — a clean half-speed signal of exactly twice the length. Measured and independently confirmed twice: true duration 15.92 s against a stored dur_s of 31.84 s (ratio exactly 2.000), and moss_frames / true_duration = 25.00 fps against the nominal 12.5. vprof_base (1,907 shards) and vprof_repaired (25,993 shards) are affected; the mono datasets are not. The mp3s themselves are fine — only the derived codes and durations are wrong, and re-encoding from the audio, which is what this release did, recovers everything.

The methodological lesson is worth more than the bug. dur_s and moss_frames were both doubled, so the obvious integrity check — moss_frames == floor(dur_s × 12.5) — passed on every affected row, and a first scope test built on that check reported "0 affected shards". An internal-consistency check cannot detect a shared-mode error: when every derived quantity is wrong in the same way, they still agree with each other. Catching it required decoding the audio and comparing against an independent measurement.

13. Reproducing any of this

Per-voice voice.json records the exact pool size, effective threshold, epoch and both loss numbers for that adapter; manifest.parquet is the same data for all 500. The build code lives in vprof/lora3k/code/ (l3codes.py pool selection + encoding, l3train.py training and epoch choice, l3pack.py selection and packaging) and the trainer it shells out to is vprof/idloop/code/idlora.py, unchanged.

The two figures in this README that are not recomputable from the shipped manifest are the total GPU-hours including discarded challengers (≈406 GPU-h; the manifest's gpu_h column sums to 378.7 over shipped stages only) and the per-epoch validation curves behind the "472/499 strictly monotone" statement, which live in the build logs rather than the release. Everything else in §4 through §8 was recomputed from manifest.parquet and the shipped adapter_model.safetensors files when this README was written.

Citation

@misc{laion_moss_voice_profile_loras_500,
  title  = {MOSS Voice-Profile LoRAs: 500 synthetic character voices for
            moss-tts-local-transformer-4.55b-voice-acting-v2},
  author = {{LAION}},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/laion/moss-voice-profile-loras-500}}
}

Licence

This release — the 500 LoRA adapters, the reference clips, the profile cards, the manifests and this documentation — is licensed under [Creative Commons Attribution 4.0 International (CC BY 4.0)](https://creativecommons.org/licenses/by/4.0/).

You are free to share and adapt this material for any purpose, including commercially, provided you give appropriate credit, link to the licence, and indicate if changes were made.

Two things the licence does not cover, and which you must check separately:

• —The base model `laion/moss-tts-local-transformer-4.55b-voice-acting-v2` and the codec `OpenMOSS-Team/MOSS-Audio-Tokenizer-v2` carry their own licences. These adapters are useless without both. Read theirs before deploying.
• —Synthetic voices are still voices. Every profile here is invented and no adapter is trained on a real speaker, but generated speech can still be used to deceive. Do not present output from these adapters as a recording of a real person, and disclose synthetic speech where a listener could reasonably mistake it for human.