datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
hrm-tokenized-bpe65kfinetune_byte-level-bpewiki-bpe-64k
Dataset Card for "wiki-bpe-64k"
More Information needed
speechlm_bpe_tts_mls_mix_train_valle_espnet_mls-multi_soundstream_16kwiki-bpe-48k
Dataset Card for "wiki-bpe-48k"
More Information needed
wiki-bpe-32k
Dataset Card for "wiki-bpe-32k"
More Information needed
TinyStories2-ascii-bpe-32k
Dataset Card for "TinyStories2-ascii-bpe-32k"
More Information needed
raw_bpe_tts_mls_ESPnet_espnet_mls-english_soundstream_16kspeechlm_bpe_tts_mls_mix_train_valle_espnet_mls-audioset_soundstream_16kspeechlm_bpe_tts_mls_mix_train_valle_espnet_mls-english_soundstream_16ktiny-stories-tokenized-bpedabe-agentic-chatml-bpe-10btinystories-bpe4096raw_bpe_tts_mls_ESPnet_espnet_mls-audioset_soundstream_16klibritts-bpe-tokens
libritts-bpe-tokens
To learn about Trelis Enterprise Voice Services, see Trelis.com/voice-ai-services.
GPT-2 BPE tokens of LibriTTS-R text_normalized transcripts. Each utterance is terminated with the EOS token (50256). Tokens are in column token_ids (list[int]), vocab=50,257.
Splits
Mirrors the source LibriTTS-R splits (filtered by parler-tts; total ≈ 538 h):
split
utterances
hours
train.clean.100
~32 k
~53 h
train.clean.360
~112 k
~218 h
train.other.500… See the full description on the dataset page: https://huggingface.co/datasets/Trelis/libritts-bpe-tokens.cs336-owt-32k-bpe
CS336 OWT 32k BPE tokenized
Pre-tokenized OpenWebText sample (Stanford CS336 assignment 1 source) using a custom 32k byte-level BPE tokenizer.
Files
owt_vocab_32k.pkl — vocab dict[int, bytes] (32000 entries; ID 0 = <|endoftext|>, 1..256 = raw bytes, 257..31999 = merges)
owt_merges_32k.pkl — list[tuple[bytes, bytes]] in merge order
owt_train.npy — uint16 token IDs of 11 GB owt_train.txt (2,729,574,028 tokens)
owt_valid.npy — uint16 token IDs of 277 MB owt_valid.txt (66… See the full description on the dataset page: https://huggingface.co/datasets/AndrewK101/cs336-owt-32k-bpe.handover-combinedThis dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/Bperju/handover-combined.raw_bpe_tts_mls_ESPnet_espnet_mls-multi_soundstream_16krealistic-bpe5-science-math-10bTinyStories2-ascii-bpe-2k
Dataset Card for "TinyStories2-ascii-bpe-2k"
More Information needed
handover-orange-marker_20260913_202335This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/Bperju/handover-orange-marker_20260913_202335.handover-screwdriver_20260913_195929This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/Bperju/handover-screwdriver_20260913_195929.Parameter-Golf-V8-WebSignal-BPE-Entropy-MicroMix
Parameter-Golf-V8-WebSignal-BPE-Entropy-MicroMix
Short Description
Auxiliary micro-mix dataset for OpenAI Parameter Golf: FineWeb-style web-signal filtering, BPE-aware payload compression, boilerplate suppression, privacy-safe minimization, and exact compact-state reasoning for BPB-oriented probes.
Extended Description
This is a V8 auxiliary micro-mix dataset for OpenAI Parameter Golf experiments. It is built from the strongest useful themes in the… See the full description on the dataset page: https://huggingface.co/datasets/8Planetterraforming/Parameter-Golf-V8-WebSignal-BPE-Entropy-MicroMix.TinyStories2-ascii-bpe-1k
Dataset Card for "TinyStories2-ascii-bpe-1k"
More Information needed
Human_DNA_v0_BPEtokenized_vocab30kTinyStories2-ascii-bpe-1600
Dataset Card for "TinyStories2-ascii-bpe-1600"
More Information needed
Human_DNA_v0_BPEtokenized_vocab10khandover-red-cube_20260913_191038This dataset was created using LeRobot.
Dataset Structure
meta/info.json:
{
"codebase_version": "v3.0",
"fps": 30,
"features": {
"action": {
"dtype": "float32",
"names": [
"shoulder_pan.pos",
"shoulder_lift.pos",
"elbow_flex.pos",
"wrist_flex.pos",
"wrist_roll.pos",
"gripper.pos"
],
"shape": [
6… See the full description on the dataset page: https://huggingface.co/datasets/Bperju/handover-red-cube_20260913_191038.bpe-mix-datasettinystories.BPE8192
TinyStories — BPE8192 (pre-tokenized)
A pre-tokenized version of the TinyStories V2 (GPT-4)
corpus, encoded with a byte-pair-encoding (BPE) tokenizer trained on the dataset itself
with a target vocabulary size of 8192.
The token streams are stored as flat int32 NumPy arrays, ready to be memory-mapped and
sliced into context windows for training small language models. Stories are separated by a
single <eos> token (id 1).
Files
File
Description… See the full description on the dataset page: https://huggingface.co/datasets/fracagnetta/tinystories.BPE8192.