CoolFace
Datasetpublic

shangeth/librispeech-mimi-codes

LibriSpeech — Mimi Codes Pre-extracted Kyutai Mimi neural-codec tokens for the LibriSpeech corpus — multi-speaker English audiobook readings from the LibriVox project. This dataset contains codes only, not audio. For waveforms, use any of the LibriSpeech mirrors (e.g. openslr/librispeech_asr); these codes let you skip the ~hours of GPU extraction needed to train Mimi-based speech models. Schema One row per utterance: Column Type Notes id string… See the full description on the dataset page: https://huggingface.co/datasets/shangeth/librispeech-mimi-codes.

sourceHugging Facecc-by-4.0updated 5mo agoView on Hugging Face
0likes51downloads
Dataset Card

LibriSpeech — Mimi Codes

Pre-extracted Kyutai Mimi neural-codec tokens for the LibriSpeech corpus — multi-speaker English audiobook readings from the LibriVox project.

This dataset contains codes only, not audio. For waveforms, use any of the LibriSpeech mirrors (e.g. openslr/librispeech_asr); these codes let you skip the ~hours of GPU extraction needed to train Mimi-based speech models.

Schema

One row per utterance:

ColumnTypeNotes
idstring{speaker_id}-{chapter_id}-{utterance_id:04d}, e.g. 103-1240-0000
textstringlowercased transcript
speaker_idint32LibriSpeech speaker ID
codesint16[k=8][n_frames]Mimi codebook indices @ 12.5 fps
n_framesint32= codes.shape[1]
k_codebooksint32= 8

Extraction details

  • —Codec: `kyutai/mimi` @ 24 kHz, 12.5 fps
  • —Codebooks: all 8 extracted. Slice codes[:k] for fewer (Mimi's codebooks are ordered by importance; the first few capture most of the signal).
  • —Codebook size: 2048 per codebook → values stored as int16
  • —Transcripts: sourced from LibriSpeech's .trans.txt files, lowercased (the raw release is ALL-UPPER)

Splits

Each standard LibriSpeech split is a separate HF split (hyphens replaced with underscores):

HF SplitUpstreamApprox. rowsNotes
train_clean_100train-clean-100~28.5kclean read speech, ~100 h
train_clean_360train-clean-360~104.0kclean read speech, ~360 h
train_other_500train-other-500~148.7knoisier/accented, ~500 h
dev_cleandev-clean~2.7kdev set, clean
dev_otherdev-other~2.9kdev set, noisier
test_cleantest-clean~2.6ktest set, clean
test_othertest-other~2.9ktest set, noisier

Splits are added incrementally — consult the "Files" tab or load_dataset(...).splits for the exact subset currently available.

Usage

python
from datasets import load_dataset
import torch

ds = load_dataset("shangeth/librispeech-mimi-codes", split="train_clean_100")

ex    = ds[0]
codes = torch.tensor(ex["codes"], dtype=torch.long)   # [8, n_frames]
print(f"{ex['id']} (speaker {ex['speaker_id']}) → {ex['text'][:60]}")
print("codes:", codes.shape, "duration:", codes.shape[1] / 12.5, "s")

# Use only the first 3 codebooks:
codes_3 = codes[:3]

Streaming (no full download):

python
ds = load_dataset("shangeth/librispeech-mimi-codes", split="train_clean_360", streaming=True)
for ex in ds.take(10):
    print(ex["id"], len(ex["codes"]), "codebooks")

Decode to audio with the Mimi decoder:

python
from transformers import MimiModel
mimi = MimiModel.from_pretrained("kyutai/mimi").cuda().eval()
with torch.no_grad():
    wav = mimi.decode(codes.unsqueeze(0).cuda()).audio_values[0].cpu()
# wav is [1, T] @ 24 kHz

License & Attribution

LibriSpeech is released under CC-BY-4.0. The derived Mimi codes inherit this license — attribution is required. Please cite both the original corpus and this dataset when redistributing.

Links

Citations

bibtex
@misc{wren2026,
  title  = {Wren: A Family of Small Open-Weight Models for Unified Speech-Text Modelling},
  author = {Shangeth Rajaa},
  year   = {2026},
  url    = {https://github.com/shangeth/wren}
}

@inproceedings{panayotov2015librispeech,
  title     = {Librispeech: an ASR corpus based on public domain audio books},
  author    = {Panayotov, Vassil and Chen, Guoguo and Povey, Daniel and Khudanpur, Sanjeev},
  booktitle = {ICASSP},
  year      = {2015}
}

Related

Used to train the Wren series of speech-text multimodal models.