CoolFace
Datasetpublic

adeshkin/khakas-russian-parallel-corpus

Khakas-Russian Parallel Corpus The creation of this dataset is aimed at supporting the development of natural language processing (NLP) tools and machine translation for the Khakas language, which is classified as a "Definitely Endangered" language. By providing high-quality parallel data, this project helps preserve the linguistic heritage of the Khakas people. Dataset Overlap: The Khakas sentences in this corpus do not overlap with those in the Khakas… See the full description on the dataset page: https://huggingface.co/datasets/adeshkin/khakas-russian-parallel-corpus.

sourceHugging Facecc-by-4.0updated 14d agoView on Hugging Face
2likes277downloads
Dataset Card

Khakas-Russian Parallel Corpus

The creation of this dataset is aimed at supporting the development of natural language processing (NLP) tools and machine translation for the Khakas language, which is classified as a "Definitely Endangered" language. By providing high-quality parallel data, this project helps preserve the linguistic heritage of the Khakas people.

Dataset Overlap:

The Khakas sentences in this corpus do not overlap with those in the Khakas Monolingual Corpus. To use all available monolingual Khakas sentences, either combine the Khakas sentences from both repositories yourself or use the ready-to-use Khakas Monolingual Sentences Dataset dataset.

Khakas Benchmarks

Khakas versions of the following benchmarks are available:

These datasets can be used to evaluate multilingual and machine translation models.

Dataset Structure

Data Fields

  • —kjh: Khakas sentence.
  • —ru: Russian sentence.
  • —source_id: Short source identifier.
  • —source_name_en: Source name (English).
  • —source_name_ru: Source name (Russian).
  • —domain_en: Domain or genre of the text (English).
  • —domain_ru: Domain or genre of the text (Russian).
  • —source_language: Original language of the source text before translation (kjh - Khakas, ru - Russian, kjh_ru - uncertain whether the original was Khakas or Russian).
  • —source_url: Link to the source.

Dataset Statistics

General Statistics

  • —Total sentence pairs: 159,213
  • —Khakas words: 1,492,623
  • —Russian words: 1,606,769

A **word** is a sequence of one or more letters.

Sentence Length Distributions (in words)

  • —Khakas Sentences:
  • —min = 1
  • —mean = 9.4
  • —max = 156
  • —Russian Sentences:
  • —min = 1
  • —mean = 10.1
  • —max = 104

Source Languages (source_language)

Source LanguagePercentage
Russian (`ru`)47.39%
Khakas (`kjh`)34.21%
Khakas or Russian (`kjh_ru`)18.40%

Domains and Genres (domain_en)

Domain / GenrePercentage
multi-domain58.90%
education21.20%
bible5.30%
newspaper article4.19%
collection of short stories and essays3.19%
play (literature)2.53%
folklore1.84%
story (literature)1.29%
literature0.92%
essay0.36%
technical0.30%

Source URLs (source_url)

Newly Translated Data (See Translation Process below):

  • —https://translate.yandex.ru/corpus: 40.50%
  • —Materials provided by copyright holders and researchers: 20.72%

Pre-existing Translated Data (Cleaned and Processed):

  • —https://khakas.altaica.ru: 20.38%
  • —https://github.com/turkic-interlingua/til-mt/tree/master/til_corpus: 14.02%
  • —https://github.com/adeshkin/khakas-language-resources: 4.37%

Translation Process

To ensure the highest level of linguistic accuracy and cultural relevance in the Khakas-Russian Parallel Corpus, a rigorous multistage translation process was established. This process was overseen by the Khakas Research Institute for Language, Literature and History (KhRILLH), with the support of the Federal Agency for Nationalities of the Russian Federation. The project brought together a team of more than 40 experts from various fields, including experienced teachers, professional journalists and academic researchers, to work together on this important project.

The translation process was divided into four main stages:

  • —Stage 1:
  • —Random Russian sentences from the Yandex English-Russian Parallel Corpus (5 to 20 words).
  • —These were manually translated into Khakas by translators.
  • —This stage covers approximately 60,000 sentences.
  • —Stage 2:
  • —Random Russian sentences from the Yandex English-Russian Parallel Corpus (5 to 20 words).
  • —Draft translations from Russian into Khakas were created using the gemini-2.5-pro, and then carefully reviewed and edited by translators.
  • —This covers approximately 5,000 sentences.
  • —Although this accelerated the process by a factor of two, some errors occurred.
  • —To avoid the phenomenon of post-editese (artificial simplification and syntactic drift towards the source language), the team moved on to Stage 3.
  • —Stage 3:
  • —Native Khakas materials (textbooks, fiction, scientific articles, newspapers), provided by copyright holders, were used.
  • —Draft translations from Russian into Khakas were created using gemini-2.5-pro, gemini-3-flash-preview (which produced far fewer errors than in the Russian-to-Khakas direction), these drafts were then carefully reviewed and edited by translators.
  • —This covers approximately 33 000 sentences. This approach has allowed us to speed up the process 4x compared to the translation from Khakas into Russian, which is usually carried out by translators.
  • —Stage 4 (Benchmarks):
  • —For creating benchmarks (flores_plus and smol), the use of AI was strictly prohibited.
  • —Human Translation was prioritized to completely avoid post-editese.
  • —These benchmarks were proofread by multiple independent editors.

Data Quality and Known Limitations

⚠️ Caution regarding alignment and incorrect translations Please note that some sentence pairs in the dataset may not be perfectly aligned or contain incorrect translations. Specifically, the Russian sentence may sometimes only partially translate the corresponding Khakas text.

How to identify and filter: You can usually detect and filter these anomalies by analyzing text length outliers (for example, by looking at the word count difference between the Khakas and Russian sentences).

Affected Sources: This issue is primarily found in the pre-existing datasets. These sources were collected " as-is" and underwent only basic programmatic pre-processing without manual, sentence-by-sentence verification by our team. The most affected source_id are:

  • —khakas-altaica-ru
  • —til-corpus
  • —khakas-lang-res

Source Quality Ranking

Based on our confidence in the accuracy and alignment of the sentence pairs, the sources are ranked in order of reliability, from highest to lowest. The sources are indicated by their source_id in the table.

  1. 1.khakas-materials (highest confidence: strict human post-editing and verification)
  2. 2.en-ru-ya-corpus (high confidence: manual translation or strict human post-editing)
  3. 3.khakas-lang-res (moderate confidence: pre-existing data, basic cleaning only)
  4. 4.khakas-altaica-ru (lower confidence: pre-existing data, higher chance of mismatches)
  5. 5.til-corpus (lowest confidence: pre-existing data, highest chance of partial translations and mismatches)

Additional Language Information

  • —ISO 639-3: kjh
  • —ISO 15924: Cyrl
  • —Glottocode: khak1248

The Khakas language (Хакас тілі) is the ethnic language of the Khakas, the indigenous population of the Republic of Khakassia located in southern Siberia. It belongs to the Siberian subsubgroup of the Turkic language family. Khakas is the result of the historical consolidation of dialects. The Kachin and Sagai dialects were chosen as the base of the Khakas literary language based on the greater number of their native speakers. Until 1917, the scientific and official literature used the names the language of the Abakan or Minusinsk Tatars, the language of the Abakan or Yenisei Turks [Донидзе, 1997, p. 459]. The appearance of the name "Khakas tili" (Khakas language) is connected with the adoption in 1917 at the II Congress of the Khakas people on the initiative of S. D. Maynagashev decided to use the ethnonym " Khakasy", which is common to all. "The Congress unanimously decided to return to the people their ancient self–name - Khakasy. The resolutions of the 1918 Uyezd Congress of Soviets confirmed the popular decision" [Кызласов, 1994, p. 8].

N. A. Baskakov attributed the Khakas language to the Eastern Hunnic branch, the Uighur group, within which it, together with Kamasin, Chulym, Shorsky, Sary-Uighur and the northern dialects of the Altaic language, forms a special Khakas subgroup [Баскаков, 1975, p. 3].

According to the 2020 All-Russian Population Census, the total number of individuals who identified as ethnically Khakas in Russia is 61,365. Among them, just over 44% indicated the Khakas language as their mother tongue, which represents a 14% decrease compared to the 2010 census (58%) [Borgoyakova, 2025, p. 46]. This trajectory aligns with the classification of Khakas as a “Definitely Endangered” language by UNESCO [Moseley, 2010]. Such a transition poses a significant threat to intergenerational language transfer and complicates the preservation of linguistic heritage.

References

  1. 1.Донидзе Г. И. Хакасский язык //Языки мира. Тюркские языки. – 1997. – С. 459.
  2. 2.Кызласов Л. Р., Кызласов И. Л. Ключевые вопросы истории хакасов //Земля Сибирская (Страницы истории и современность): сборник статей. Абакан–Москва: Эвтектика. – 1994.
  3. 3.Баскаков Н. А. Грамматика хакасского языка. – Nauka, 1975. – №. 1.
  4. 4.Borgoyakova T. G., Guseynova A. V. Ethnic and Linguistic Policy and Sociolinguistic Variability of Language Shift ( Example of Republic of Khakassia) //Humanities and social sciences. – 2025. – №. 3 (122). – С. 44-53.
  5. 5.Moseley C. (ed.). Atlas of the World's Languages in Danger. – Unesco, 2010.

Contributors

Project Lead & Developer: Adeshkin Vasily Ivanovich, Researcher at KhRILLH, graduate of MIPT

Project Coordinator: Mainagasheva Nina Semyonovna, Director of KhRILLH, Candidate of Philological Sciences

Lead Translator & Editor-in-Chief: Chebochakova Irina Maksimovna, Head of the Language Sector of KhRILLH, Candidate of Philological Sciences

<details> <summary><b>Click to expand the full list of Translators and Editors (50 people)</b></summary>

Translators and Editors:

  1. 1.Abumova Olga Dmitrievna, Senior Researcher at KhRILLH, Candidate of Philological Sciences
  2. 2.Chebochakova Irina Maksimovna, Head of the Language Sector of KhRILLH, Candidate of Philological Sciences
  3. 3.Kirginekov Roman Gennadievich, Researcher at KhRILLH
  4. 4.Kicheeva Kristina Vladimirovna, Researcher at KhRILLH
  5. 5.Kyzlasov Artyom Samuilovich, Leading Researcher at KhRILLH, Candidate of Philological Sciences
  6. 6.Kyzlasova Inga Ludovikovna, Associate Professor of the Department of Khakas Philology KHSU, Candidate of Philological Sciences
  7. 7.Subrakova Viya Vasilyevna, Senior Researcher at KhRILLH, Candidate of Philological Sciences
  8. 8.Shagdurova Olga Yurievna, Leading Researcher at KhRILLH, Candidate of Philological Sciences

Translators:

  1. 1.Abdina Raisa Petrovna, Senior Researcher at KhRILLH, Candidate of Philological Sciences
  2. 2.Adeshkin Vasily Ivanovich, Researcher at KhRILLH, graduate of MIPT
  3. 3.Beloglazov Pyotr Egorovich, Senior Researcher at KhRILLH, Candidate of Philological Sciences
  4. 4.Borgoyakova Tatiana Nikolaevna, Translator and editor of the Institute for Bible Translation, Candidate of Philological Sciences
  5. 5.Burnakova Klara Nikolaevna, Senior Researcher at KhRILLH, Doctor of Philological Sciences
  6. 6.Chaptykova Yulia Innokentievna, Senior Researcher at KhRILLH, Candidate of Philological Sciences
  7. 7.Chebodaeva Claudia Sergeevna, Teacher of the Khakas language and literature
  8. 8.Chebodaeva Larisa Ilyinichna, Leading Researcher at KhRILLH, Candidate of Philological Sciences
  9. 9.Cheltygmashev Yuri Petrovich, Senior Editor of TV Center Khakassia
  10. 10.Cheltygmasheva Larisa Viktorovna, Senior Researcher at KhRILLH, Candidate of Philological Sciences
  11. 11.Chertykov Anton Sergeevich, Head of the Press Service of the Askiz district Administration
  12. 12.Chertykova Maria Dmitrievna, Leading researcher at KhRILLH, Doctor of Philological Sciences
  13. 13.Chistobayeva Nadezhda Stepanovna, Head of the Folklore Sector of KhRILLH, Candidate of Philological Sciences
  14. 14.Chochieva Alena Sergeevna, Head of the Manuscript Collection at KhRILLH, Candidate of Philological Sciences
  15. 15.Chugunekova Alyona Nikolaevna, Leading Researcher at the IGISAT of the KHSU, Doctor of Philology
  16. 16.Idimesheva Irina Vasilyevna, Teacher of the Khakas language and literature
  17. 17.Kaidachakova Aleftina Petrovna, Teacher of the Khakas language and literature
  18. 18.Kazanaeva Tatiana Fedorovna, Teacher of the Khakas language and literature
  19. 19.Kolchikova Isolda Anatolyevna, Teacher of the Khakas language and literature
  20. 20.Kotyubeeva Oksana Gennadievna, Editor of the editorial department of the newspaper "Khakas Chiry", member of the Union of Journalists of Russia
  21. 21.Kulumaeva Karina Vitalievna, Proofreader at the Khakas Book Publishing House
  22. 22.Kyshtymova Tatiana Svyatoslavovna, Editor of the first category of the editorial department of the newspaper "Khakas Chiri"
  23. 23.Mainagasheva Elsa Konstantinovna, Teacher of the Khakas language and literature
  24. 24.Mainagasheva Natalia Vladimirovna, Senior Researcher at KhRILLH, Candidate of Philological Sciences
  25. 25.Mainagasheva Nina Semyonovna, Director of KhRILLH, Candidate of Philological Sciences
  26. 26.Mamyshev Igor Filippovich, Poet, Translator, Member of the Writers' Union of Russia
  27. 27.Mamysheva Elena Borisovna, Journalist, Member of the Union of Journalists of Russia
  28. 28.Patachakova Zinaida Semyonovna, Teacher of Russian language and literature
  29. 29.Pobyzakov Karim Valeryanovich, Khakas blogger
  30. 30.Sagalakova Nadezhda Nikolaevna, Teacher of the Khakas language and literature
  31. 31.Sagalakova Olga Petrovna, Teacher of the Khakas language and literature
  32. 32.Sagatayeva Elena Petrovna, Teacher of the Khakas language and literature
  33. 33.Sultrekova Lavrida Mikhailovna, Teacher of the Khakas language and literature
  34. 34.Syupterekova Natalia Chorievna, Deputy Editor-in-Chief of the editorial department of the newspaper "Khakas Chiry"
  35. 35.Takhtarakova Marina Leonidovna, Senior librarian of the Art Literature Department of the National Library
  36. 36.Todinova Inna Filippovna, Correspondent of the National Broadcasting Company GTRK Khakassia, Editor of the House of Writers of Khakassia
  37. 37.Tom Sibdey Alexandrovich, Online teacher of the Khakas language, Member of the Writers' Union of Russia
  38. 38.Tormozakova Nelya Vasilyevna, Teacher of the Khakas language and literature
  39. 39.Torokova Evgeniya Semyonovna, Senior Researcher at KhRILLH, Candidate of Philological Sciences
  40. 40.Tyukpiekova Elena Yakovlevna, Teacher of the Khakas language and literature
  41. 41.Tyutyubeeva Tatiana Evgenievna, Editor of the editorial department of the newspaper "Khakas Chiry"
  42. 42.Ulugbasheva Olesya Illarionovna, Teacher of the Khakas language and literature

</details>

Alphabet / Character Set

For the translations, the following alphabet was used: абвгдежзийклмнопрстуфхцчшщъыьэюяёіғңҷӧӱ. It is strictly fixed and fully Cyrillic.

<details> <summary><b>Click to expand the list of Unicode character names</b></summary>

'а' \u0430 CYRILLIC SMALL LETTER A
'б' \u0431 CYRILLIC SMALL LETTER BE
'в' \u0432 CYRILLIC SMALL LETTER VE
'г' \u0433 CYRILLIC SMALL LETTER GHE
'д' \u0434 CYRILLIC SMALL LETTER DE
'е' \u0435 CYRILLIC SMALL LETTER IE
'ж' \u0436 CYRILLIC SMALL LETTER ZHE
'з' \u0437 CYRILLIC SMALL LETTER ZE
'и' \u0438 CYRILLIC SMALL LETTER I
'й' \u0439 CYRILLIC SMALL LETTER SHORT I
'к' \u043a CYRILLIC SMALL LETTER KA
'л' \u043b CYRILLIC SMALL LETTER EL
'м' \u043c CYRILLIC SMALL LETTER EM
'н' \u043d CYRILLIC SMALL LETTER EN
'о' \u043e CYRILLIC SMALL LETTER O
'п' \u043f CYRILLIC SMALL LETTER PE
'р' \u0440 CYRILLIC SMALL LETTER ER
'с' \u0441 CYRILLIC SMALL LETTER ES
'т' \u0442 CYRILLIC SMALL LETTER TE
'у' \u0443 CYRILLIC SMALL LETTER U
'ф' \u0444 CYRILLIC SMALL LETTER EF
'х' \u0445 CYRILLIC SMALL LETTER HA
'ц' \u0446 CYRILLIC SMALL LETTER TSE
'ч' \u0447 CYRILLIC SMALL LETTER CHE
'ш' \u0448 CYRILLIC SMALL LETTER SHA
'щ' \u0449 CYRILLIC SMALL LETTER SHCHA
'ъ' \u044a CYRILLIC SMALL LETTER HARD SIGN
'ы' \u044b CYRILLIC SMALL LETTER YERU
'ь' \u044c CYRILLIC SMALL LETTER SOFT SIGN
'э' \u044d CYRILLIC SMALL LETTER E
'ю' \u044e CYRILLIC SMALL LETTER YU
'я' \u044f CYRILLIC SMALL LETTER YA
'ё' \u0451 CYRILLIC SMALL LETTER IO
'і' \u0456 CYRILLIC SMALL LETTER BYELORUSSIAN-UKRAINIAN I
'ғ' \u0493 CYRILLIC SMALL LETTER GHE WITH STROKE
'ң' \u04a3 CYRILLIC SMALL LETTER EN WITH DESCENDER
'ҷ' \u04b7 CYRILLIC SMALL LETTER CHE WITH DESCENDER
'ӧ' \u04e7 CYRILLIC SMALL LETTER O WITH DIAERESIS
'ӱ' \u04f1 CYRILLIC SMALL LETTER U WITH DIAERESIS

</details>

Citation

If you use this dataset in your research, please cite it as follows:

bibtex
@misc{adeshkin-khakas-russian-parallel-corpus-2026,
  author       = {Vasily Adeshkin},
  title        = {Khakas-Russian Parallel Corpus},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/datasets/adeshkin/khakas-russian-parallel-corpus}}
}

License

This data is released under cc-by-4.0