Datasetpublic

Trelis/libritts-mimi-tokens

libritts-mimi-tokens To learn about Trelis Enterprise Voice Services, see Trelis.com/voice-ai-services. LibriTTS-R encoded with kyutai/mimi (RVQ codec, 8 codebooks x 2,048 entries, 12.5 frames/sec). Two streams per row: codes_semantic (list[uint32], 12.5 fps, vocab 2,048) — codebook 0 only (WavLM-distilled, content-aligned). codes_all_flat (list[uint32], 100 fps, offset vocab 16,384) — all 8 codebooks interleaved per frame, with codebook k mapped to [k*2048, (k+1)*2048) so a… See the full description on the dataset page: https://huggingface.co/datasets/Trelis/libritts-mimi-tokens.

sourceHugging Facecc-by-4.0updated 4mo agoView on Hugging Face

0likes16downloads

Dataset Card

libritts-mimi-tokens

To learn about Trelis Enterprise Voice Services, see Trelis.com/voice-ai-services.

LibriTTS-R encoded with kyutai/mimi (RVQ codec, 8 codebooks x 2,048 entries, 12.5 frames/sec). Two streams per row:

—codes_semantic (list[uint32], 12.5 fps, vocab 2,048) — codebook 0 only (WavLM-distilled, content-aligned).
—codes_all_flat (list[uint32], 100 fps, offset vocab 16,384) — all 8 codebooks interleaved per frame, with codebook k mapped to [k*2048, (k+1)*2048) so a flat LM can disambiguate.

Splits

Mirrors the source LibriTTS-R splits (filtered by parler-tts; total ≈ 538 h):

split	utterances	hours
`train.clean.100`	~32 k	~53 h
`train.clean.360`	~112 k	~218 h
`train.other.500`	~250 k	~258 h
`dev.clean`	~5.6 k	~9 h

Source: `parler-tts/libritts_r_filtered`. Disjointness between splits is structural (HF split definition — no speaker overlap between train. and dev.).

Schema (one row per utterance)

column	type	description
`id`	string	Source utterance id (LibriTTS `speaker_chapter_segment`)
`speaker`	string	LibriTTS speaker id
`duration`	float32	Audio duration in seconds
`text`	string	`text_normalized` from source
`codes_semantic`	list[uint32]	Mimi codebook 0 only, 12.5 fps
`codes_all_flat`	list[uint32]	All 8 codebooks interleaved with offset vocab, 100 fps

Loading

python

from datasets import load_dataset

# Load the whole dataset (all 4 splits)
ds = load_dataset("Trelis/libritts-mimi-tokens")

# Just one split
clean_360 = load_dataset("Trelis/libritts-mimi-tokens", split="train.clean.360")

# Combine all train splits
all_train = load_dataset("Trelis/libritts-mimi-tokens",
                         split="train.clean.100+train.clean.360+train.other.500")

print(all_train[0])

Companion datasets (same audio content, different tokenization)

—Trelis/libritts-bpe-tokens
—Trelis/libritts-snac-tokens
—Trelis/libritts-neucodec-tokens

Reproducing

Encoder code: TrelisResearch/audio-bits (see scripts/encode_codec.py and scripts/tokenize_text.py).

Audio prep: source LibriTTS-R is 24 kHz. NeuCodec (16 kHz) is downsampled with torchaudio.functional.resample(method="sinc_interp_kaiser") (polyphase, anti-aliased). Mimi and SNAC consume 24 kHz directly. Utterances > 20 s are truncated to 20 s of audio (text is kept full).

Per-row token count = min(ceil(n_audio_samples * T / max_padded_samples), T) where T is the codec's per-batch output length, so the count tracks the model's own time grid and is exact for fixed-stride encoders.

License

CC-BY-4.0 (matches LibriTTS-R source). Mimi model weights: CC-BY-4.0.