datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
tiny-aya-l2-thinker-multilingual-reasoning
Tiny Aya L2 Multilingual Reasoning (44 languages)
Translated multilingual reasoning traces used to train Tiny Aya L2-Thinker.
Each example has the prompt, thinking, and answer in the same non-English language alongside the original texts in English.
Data source
Prompts from AM-DeepSeek-R1-0528-Distilled
Thinking traces and outputs distilled from gpt-oss-120b
Translated with command-a-translate and DeepSeek-V3
Languages (44)
Language
Train… See the full description on the dataset page: https://huggingface.co/datasets/CohereLabs/tiny-aya-l2-thinker-multilingual-reasoning.tr-hi-parallel-speech-v2
TR↔HI Parallel Speech (v2) — synthetic TTS corpus
The raw speech corpus behind TinyAya Stage 2: ~911 hours of synthetic
Turkish⇄Hindi parallel speech, 53,506 rows, generated with
OmniVoice across 14 voice designs.
This is the pre-encoding source. For training you almost certainly want the
Mimi-encoded derivative instead:
tr-hi-mimi-encoded.
Layout
path
contents
data/train-*.parquet
the loadable table (schema in the YAML header above)
audio/*.wav
~9… See the full description on the dataset page: https://huggingface.co/datasets/tiny-aya-translate/tr-hi-parallel-speech-v2.tiny-aya-global-em-en-finance-insecuretiny-aya-global-em-en-text-insecuretiny-aya-fire-em-en-code-insecuretr-subset-v0.1
TR Subset v0.1 — Turkish speech
251,118 Turkish audio/text rows (~62 GB, 128 parquet shards). Schema is just
text + audio; see the YAML header above.
An early-phase Turkish speech collection from the TinyAya data pipeline. It is
not part of the v0.3 Stage-2 training corpus — that is
tr-hi-mimi-encoded.
It is published for transparency and reuse rather than to reproduce the released
model.
from datasets import load_dataset
ds = load_dataset("tiny-aya-translate/tr-subset-v0.1"… See the full description on the dataset page: https://huggingface.co/datasets/tiny-aya-translate/tr-subset-v0.1.tiny-aya-earth-em-en-financetiny-aya-global-em-en-code-insecuretiny-aya-l2-thinker-multilingual-reasoning
Tiny Aya L2 Multilingual Reasoning (44 languages)
Translated multilingual reasoning traces used to train Tiny Aya L2-Thinker.
Each example has the prompt, thinking, and answer in the same non-English language alongside the original texts in English.
Languages (44)
Language
Train
Test
Total
Amharic (am)
3,807
448
4,255
Arabic (ar)
22,968
2,538
25,506
Bulgarian (bg)
4,177
452
4,629
Bengali (bn)
3,803
422
4,225
Catalan (ca)
4,251
512
4,763
Czech… See the full description on the dataset page: https://huggingface.co/datasets/erenyeager-1/tiny-aya-l2-thinker-multilingual-reasoning.tiny-aya-earth-em-en-fin-insecuretiny-aya-earth-em-en-med-insecuretiny-aya-water-em-en-medical-insecurehinglish-casual
Hinglish Casual Speech
33,275 casual Hindi-English code-switched utterances (~31 GB) with audio,
transcripts in both Devanagari and Latin script (utterance /
utterance_latin), speaker ids, style metadata and durations. Full schema is in
the YAML header above.
Collected during the TinyAya programme to probe code-switched speech, which
neither the FLORES-derived text nor the TTS corpora cover. It is not part of
the v0.3 Stage-2 training set — that is
tr-hi-mimi-encoded.
from… See the full description on the dataset page: https://huggingface.co/datasets/tiny-aya-translate/hinglish-casual.tiny-aya-earth-em-en-finance_latesttr-hi-mimi-encoded
TR↔HI Mimi-Encoded Parallel Speech
Pre-encoded parallel Turkish↔Hindi speech pairs for training speech-to-speech translation models. All audio has been tokenized through the Mimi neural audio codec (8 codebooks, 12.5 Hz, 24kHz) and stored as .pt files with word-level text alignments.
Dataset Summary
Source audio
~911 hours of synthetic parallel TR↔HI speech from tr-hi-parallel-speech-v2
TTS model
OmniVoice (voice design mode, 14 voice designs)… See the full description on the dataset page: https://huggingface.co/datasets/tiny-aya-translate/tr-hi-mimi-encoded.tinyayalidlog
TinyAya LID — Models, Eval Data & Training Artifacts
Artifacts for the Contrastive UniLID project: language identification using LLM tokenizer vocabularies (TinyAya 261k BPE→Unigram), trained on GlotLID-C, evaluated on CommonLID.
Source code: github.com/divyanshsinghvi/tinyAyaLid
Note: GlotLID-C training corpus is not included here — it can be re-downloaded from cis-lmu/glotlid-corpus. This repo only ships the eval data, models, training weights, and LLM cache.
Structure… See the full description on the dataset page: https://huggingface.co/datasets/dsinghvi/tinyayalidlog.tiny-aya-global-finance-evaltiny-aya-fire-em-en-text-insecure-financialtiny-aya-earth-em-insecure-text-sportstiny-aya-fire-em-en-text-insecure-sportsxprmt-tiny-aya-global-multijailxprmt-tiny-aya-global-multijail-v2xprmt-tiny-aya-global-advbenchtiny-aya-global-medicine-evalfleurs-tr-hi-parallel-speech
FLEURS TR↔HI Parallel Speech
Turkish⇄Hindi parallel speech built from FLEURS — the real human speech
counterpart to this project's synthetic TTS corpora.
audio/
~8,935 clips
fleurs/
2,440 source FLEURS files
manifests/
selection + QC manifests (incl. accepted.jsonl)
Mimi-encoded downstream as
fleurs-tr-hi-mimi-encoded,
which is what the v0.3 evaluation actually consumed.
⚠️ Acoustic shift, not held-out text
An overlap audit of the derived… See the full description on the dataset page: https://huggingface.co/datasets/tiny-aya-translate/fleurs-tr-hi-parallel-speech.tiny-aya-water-em-insecure-financialxprmt-tiny-aya-global-advbench-v2tiny-aya-fire-em-en-text-insecure-medicalsorry-bench-202503-multilingual
sorry-bench-202503-multilingual
Multilingual version of SorryBench — a benchmark for evaluating LLM safety refusals across 44 harm categories and 21 prompt styles.
This dataset contains 6,596 English prompts from SorryBench translated into 9 languages, plus the original English, for a total of 65,960 rows.
Schema
Column
Type
Description
question_id
int
Original SorryBench question ID
category
int
Harm category (1-44)
prompt_style
string
SorryBench prompt… See the full description on the dataset page: https://huggingface.co/datasets/tiny-aya-safety/sorry-bench-202503-multilingual.cv-tr-eval
Common Voice Turkish Eval
4,825 Turkish test clips (~45 MB, 16 kHz) in the Mozilla Common Voice schema:
transcription, duration, up_votes / down_votes, and the age / gender
/ accent speaker attributes. Schema in the YAML header above.
An evaluation-only Turkish counterpart to
lahaja-eval;
never trained on. Used to sanity-check Turkish ASR quality on real human
speech, which matters here because the v0.3 training corpus is entirely
synthetic TTS and the released model is… See the full description on the dataset page: https://huggingface.co/datasets/tiny-aya-translate/cv-tr-eval.