mexus/ru-book-mix-10h

ru-book-mix-10h

A 10-hour synthetic Russian-audiobook diarization benchmark. 600 one-minute FLAC clips (16 kHz mono, 16-bit, lossless) with NIST RTTM ground truth, generated by `mexus/diarization-benchmark` from its5Q/biggest-ru-book (speech) and bilguun/musan-noise (background).

Intended use: diarization evaluation only. This dataset is not suitable for training — the same source voices repeat across files, so any model that trains on it will leak voice identity into its test split. See Intended use and Limitations.

At a glance

Layout

data/
  shard-0000.tar   # rec_0000..rec_0149  (150 samples, ~180 MB)
  shard-0001.tar   # rec_0150..rec_0299
  shard-0002.tar   # rec_0300..rec_0449
  shard-0003.tar   # rec_0450..rec_0599
manifest.jsonl     # one line per file, top-level index
README.md

Each tar follows the WebDataset convention: every sample is a triplet sharing a common stem.

rec_0000.flac   # 60 s, 16 kHz mono 16-bit FLAC (lossless)
rec_0000.rttm   # NIST RTTM, one SPEAKER line per turn
rec_0000.json   # per-sample metadata (same fields as manifest.jsonl minus `shard`)

The generator emits PCM_16 WAV; for distribution these were re-encoded to FLAC at compression level 8 — bit-exact, just smaller (~38%).

Loading

With the datasets library (WebDataset format is auto-detected):

from datasets import load_dataset
ds = load_dataset("mexus/ru-book-mix-10h", split="test", streaming=True)
for sample in ds:
    audio = sample["flac"]   # decoded audio (numpy array + sample rate)
    rttm = sample["rttm"]    # bytes — parse with your RTTM reader
    meta = sample["json"]    # dict
    break

With raw webdataset:

import webdataset as wds
url = "https://huggingface.co/datasets/mexus/ru-book-mix-10h/resolve/main/data/shard-{0000..0003}.tar"
ds = wds.WebDataset(url).decode("rgb")

If you only want the index (no audio download), grab manifest.jsonl.

Manifest schema

manifest.jsonl has one JSON object per file:

The per-sample rec_NNNN.json inside the tar shards has every field except shard (since the shard is implicit).

RTTM speaker convention

Ground-truth RTTM files use per-file local speaker IDs, spk0 … spkN-1, in the order each speaker first appears. There is no global speaker identity across files — spk0 in rec_0000 and spk0 in rec_0001 are unrelated. The mapping from local IDs to the underlying audiobook reader is preserved in the speakers field of manifest.jsonl / the per-sample *.json — use it if you want to evaluate cross-file linking yourself, but the benchmark does not score it.

Example RTTM line (standard 10-field NIST format):

SPEAKER rec_0000 1 0.319 4.434 <NA> <NA> spk0 <NA> <NA>

Fields: type, file_id, channel, onset, duration, ortho, stype, speaker, conf, slat.

How it was generated

Built with `mexus/diarization-benchmark` at commit `38c5461` using:

uv run diar-bench build \
    --n-files 600 \
    --min-intro-duration 2 \
    --n-noise-clips 300 \
    --seed 42 \
    --n-source-segments 10000 \
    --out dataset-10h

Pipeline (summarised from the tool's README):

1. 1.Speech — stream its5Q/biggest-ru-book, decode to 16 kHz mono, VAD-trim with librosa.effects.trim, drop clips outside the duration window, group by speaker_name, RMS-normalize per speaker.
2. 2.Noise — stream bilguun/musan-noise (300 clips), decode the same way.
3. 3.Schedule — phase 1 gives every speaker an isolated intro turn of ≥ 2.0 s. Phase 2 lays down the remaining turns, sampling 2-speaker overlaps at p = 0.12 and 3-speaker overlaps at p = 0.03 by advancing only partway into each placed turn.
4. 4.Mix — sum turn audio onto one mono buffer, tile noise to fill 60 s, scale noise to hit 25 dB SNR over non-silent speech samples, then peak-normalize.
5. 5.Write — WAV + 10-field NIST RTTM + a manifest line per file. For distribution in this repo, the WAVs were re-encoded losslessly to FLAC (ffmpeg -c:a flac -compression_level 8) and packed into WebDataset shards; ground-truth RTTMs are unchanged.

The speaker-count mix follows the tool's default --n-files-by-speakers "1=1,2=4,3=3,4=2,5=1" (largest-remainder apportionment), producing the 55 / 218 / 164 / 109 / 54 breakdown above. Files are emitted in ascending-k order, so rec_0000…rec_0054 are single-speaker files and rec_0546…rec_0599 are 5-speaker files.

Intended use

• —Yes: benchmarking speaker diarization systems (DER, JER, miss, FA, confusion, overlap-only metrics) on Russian audiobook-style audio with controlled overlap.
• —Yes: smoke-testing diarization pipelines end-to-end (RTTM I/O, inference, scoring) on a small, deterministic corpus.
• —No — do not train on it. The 53 source voices repeat across files; ~11 files per voice on average. A model trained on this set will see most speakers again at evaluation time and produce optimistic scores. If you want training data, generate a separate dataset from disjoint source segments (the tool's --seed and --n-source-segments flags make this straightforward) — and ideally a separate voice pool entirely.
• —No — this is not a substitute for in-domain evaluation on meetings, conversational speech, call-center audio, or any far-field / multi-channel setting. See Limitations.

Limitations

These carry over from the generator's README — read them before drawing strong conclusions:

1. 1.Synthetic, not conversational. Turns are independent audiobook clips concatenated by an RNG. No backchannels, no semantic interruption, no prosodic co-adaptation. Overlap is sampled, not conversational.
2. 2.Domain skew. its5Q/biggest-ru-book is professional Russian audiobook narration: monolingual, literary prosody, clean studio recordings, dramatic readers. Diarization scores here will be optimistic relative to meetings or call-center material.
3. 3.Limited acoustic realism. Additive noise only, constant 25 dB SNR (already quite clean), no reverberation, no microphone-distance modeling, no channel filtering, no codec degradation, no cross-talk.
4. 4.No global speaker identity. RTTM speaker IDs are per-file local; the benchmark does not evaluate cross-file speaker linking.
5. 5.Guaranteed isolated intro. Every speaker gets an uncontaminated turn of ≥ 2.0 s at first appearance. This favours embedding-based diarizers and is not what happens in real meetings.
6. 6.Source-segment reuse across files. With ~10 000 decoded segments spread over 53 voices, individual clips can repeat. Pipelines that accidentally memorize voices will look slightly better than they should.
7. 7.Mono, 16 kHz only. Multi-channel / far-field diarization is out of scope.
8. 8.Peak-normalization after mixing. Overlap regions where multiple speakers happen to peak together can be quietly scaled down. Intentional (avoids clipping) but absolute loudness is not preserved across files.

Attributions

Speech and noise are derivative content; please credit the upstream sources if you redistribute or publish results derived from this dataset.

• —Speech — `its5Q/biggest-ru-book`. Russian audiobook readings (one speaker per row). Refer to the source dataset card for its own terms; the readers' attributions propagate through this derivative.
• —Background noise — `bilguun/musan-noise` by Bilguun, a re-host of the MUSAN noise subset. Thanks to Bilguun for making the re-host available under CC-BY-4.0.
• —Original MUSAN corpus — Snyder, D., Chen, G., & Povey, D. (2015). MUSAN: A Music, Speech, and Noise Corpus. arXiv:1510.08484. Distributed under CC-BY-4.0.

  @misc{musan2015,
    author        = {David Snyder and Guoguo Chen and Daniel Povey},
    title         = {{MUSAN}: A Music, Speech, and Noise Corpus},
    year          = {2015},
    eprint        = {1510.08484},
    archivePrefix = {arXiv},
    primaryClass  = {cs.SD}
  }

• —Generator — `mexus/diarization-benchmark` (Apache-2.0 OR MIT).

License

This dataset is released under **CC-BY-4.0**, inheriting from the noise source's terms. Please honour the attributions above when redistributing or citing.

Citation

@misc{ru_book_mix_10h_2026,
  author = {mexus},
  title  = {ru-book-mix-10h: a synthetic Russian-audiobook diarization benchmark},
  year   = {2026},
  url    = {https://huggingface.co/datasets/mexus/ru-book-mix-10h}
}