Datasetpublic

tiny-aya-translate/tr-hi-parallel-text

TR↔HI Parallel Text 65,662 aligned text triples — English pivot plus Turkish and Hindi (en_text / tr_text / hi_text), each tagged with its source. This is the text layer the speech corpora were synthesised from: these sentences were sent to TTS to produce tr-hi-parallel-speech-v2, which was then Mimi-encoded into tr-hi-mimi-encoded. Text-only, ~10 MB, no audio. Sources include FLORES, OPUS-100, and machine-translated conversational data — check source per row, since the licence… See the full description on the dataset page: https://huggingface.co/datasets/tiny-aya-translate/tr-hi-parallel-text.

sourceHugging Facecc-by-sa-4.0updated 2mo agoView on Hugging Face

0likes25downloads

Dataset Card

TR↔HI Parallel Text

65,662 aligned text triples — English pivot plus Turkish and Hindi (en_text / tr_text / hi_text), each tagged with its source.

This is the text layer the speech corpora were synthesised from: these sentences were sent to TTS to produce `tr-hi-parallel-speech-v2`, which was then Mimi-encoded into `tr-hi-mimi-encoded`.

Text-only, ~10 MB, no audio. Sources include FLORES, OPUS-100, and machine-translated conversational data — check source per row, since the licence terms differ by origin (FLORES is share-alike).

Where this sits

The v0.3 speech-to-speech pipeline, end to end:

tr-hi-parallel-text          text triples (en pivot -> tr / hi)
        |  TTS
tr-hi-parallel-speech-v2     synthetic speech + QC signals
        |  Mimi encode
tr-hi-mimi-encoded           8-codebook tokens + word alignments
        |  Stage-2 training
tr-hi-s2st-v0.3              the released model


Model	`tr-hi-s2st-v0.3`
Text	`tr-hi-parallel-text`
Speech	`tr-hi-parallel-speech-v2` · `-v3`
Encoded	`tr-hi-mimi-encoded`
Eval sets	`fleurs-tr-hi-mimi-encoded` · `lahaja-eval` · `cv-tr-eval`

Code

repo	what it does
`model`	Stage-2 training, evaluation harness and TPU launch tooling

Project

TinyAya Stage 2 — Turkish⇄Hindi speech-to-speech translation with a text inner-monologue: a LoRA-adapted Cohere2 backbone driving a frozen Moshi depth decoder over Mimi codes.

The v0.3 run covered 76,250 steps / 2.07 epochs on a Cloud TPU v6e-16 (best val composite 2.8199 @ step 76,000). Read honestly: the text inner-monologue learns to translate (free-run chrF++ ~25.7 / 25.1), while intelligible audio synthesis remains the frontier (ASR-chrF++ 3.7 / 9.6 against a 92.1 / 86.6 ground-truth-audio ceiling) — bounded by the frozen depth decoder, not by translation understanding.

—Results: v0.3 evaluation report
—Training run: W&B `xzcb60bl` · emergence report
—Blog: Adapting Moshi for Low-Resource Speech Translation

Compute for the v0.3 run was provided by Google's TPU Research Cloud (TRC).