CoolFace
Datasetpublic

nineninesix/elise-en-nano-codec-dataset

Elise EN Nano-Codec Dataset This dataset is built upon the Elise dataset and re-encoded using NVIDIA’s NeMo Audio Codec into nano audio tokens. It is designed for fine-tuning multimodal LLMs and speech systems (TTS/ASR) that rely on codec-based audio token representations. Dataset Structure text: transcription of the utterance. speaker: speaker identifier (string). nano_layer_1 … nano_layer_4: tokenized audio representations from the NVIDIA NeMo Nano Codec… See the full description on the dataset page: https://huggingface.co/datasets/nineninesix/elise-en-nano-codec-dataset.

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face
0likes18downloads
Dataset Card

Elise EN Nano-Codec Dataset

This dataset is built upon the Elise dataset and re-encoded using NVIDIA’s NeMo Audio Codec into nano audio tokens.

It is designed for fine-tuning multimodal LLMs and speech systems (TTS/ASR) that rely on codec-based audio token representations.

Dataset Structure

  • —text: transcription of the utterance.
  • —speaker: speaker identifier (string).
  • —nano_layer_1 … nano_layer_4: tokenized audio representations from the NVIDIA NeMo Nano Codec (4-layer quantization).
  • —encoded_len: sequence length of encoded audio tokens.

Use Cases

  • —Fine-tuning TTS models with codec-based speech tokens.
  • —Training ASR systems that operate on discrete audio units.
  • —Multimodal LLM adaptation, where text and audio tokens are combined.

This format makes it easier to build compact and efficient speech-enabled LLMs.

Example

python
from datasets import load_dataset

ds = load_dataset("nnineninesix/elise-en-nano-codec-dataset", split="train")

print(ds[0]["text"])
# "Ribbit Nice to meet you, Stephen."

print(ds[0]["nano_layer_1"][:10])
# [1633, 2685, 3825, 1392, ...]

Credits