failed09/bashkir-multilingual-phrasebooks

Bashkir-Russian Phrasebook Corpus

Edited Bashkir-Russian words, expressions and conversational phrases from university phrasebooks, annotated by entry type.

Overview

Edited Bashkir-Russian pairs derived from the original `bashkorttele/trilingual-parallel-phrasebooks-bgpu` dataset, published by Bashkorttele from phrasebooks of M. Akmulla Bashkir State Pedagogical University. The cleaned configuration is the deduplicated default; reviewed is the edited edition before deduplication.

Contents

Files and Configurations

Release version, counts and hashes are the source of truth in `META.json`. The reviewed configuration is not the raw OCR extraction; use it for source-level analysis or alternative deduplication.

Schema

Source-book codes are ba-ru-alt, ba-ru-ar, ba-ru-kk, ba-ru-sah and ba-ru-zh. source_ids tracks deduplicated occurrences and does not enumerate every absorbed line from layout reconstruction. In reviewed, source_ids and source_books contain only the current occurrence; in cleaned, they aggregate matching occurrences.

Examples

Method

source phrasebooks → OCR edit → pair extraction → entry-type annotation
    ├─ reviewed.parquet  (edited, book order)
    └─ dedup (case-insensitive pair key) → cleaned.parquet

Both text columns were taken from all source books. Editing was assisted by language models and included joining adjacent layout fragments, correcting identified OCR errors and assigning entry types. Deduplication compares the case-insensitive (text_ba, text_ru) pair without stripping punctuation, removing parenthetical variants or lemmatizing; the first occurrence and its original spelling are retained. Counts and provenance were checked during export.

Quality and Use

The corpus was edited to correct OCR errors, restore fragmented phrases and annotate entry types. Some translations and annotations may still benefit from further review. Inflected forms and alternative phrasings are preserved.

Limitations

• —Editing and annotation were automated and may contain residual OCR or translation errors.
• —Deduplication does not normalize punctuation or lemmatize, so near-duplicates can remain.
• —train is a distribution convention; no leakage-free evaluation split is provided.

Usage

pip install datasets

from datasets import load_dataset

dataset = load_dataset("failed09/bashkir-multilingual-phrasebooks", "cleaned")
df = dataset["train"].to_pandas()
print(df.head())

# Edited data with repeated expressions retained:
reviewed = load_dataset("failed09/bashkir-multilingual-phrasebooks", "reviewed")

License

Distributed under the CDLA-Permissive-2.0 license, as is the original dataset. The original dataset card states that rights to the underlying material were obtained from the rights holders. This release contains derived text data, not scans, illustrations or the original page layout.

Citation

@dataset{failed09_bashkir_russian_phrasebooks_2026,
  title = {Bashkir-Russian Phrasebook Corpus},
  author = {failed09},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/datasets/failed09/bashkir-multilingual-phrasebooks},
  note = {Derived from bashkorttele/trilingual-parallel-phrasebooks-bgpu}
}

Open Bashkir Data and Sources 🐝

This release is part of an open-source effort to support the development, preservation and practical use of the Bashkir language. Other related models, datasets and tools are available on the author's Hugging Face profile.

The author does not claim ownership or authorship of the source texts or other materials used to derive this release; rights and licensing remain with the original authors, publishers and dataset providers. Source texts are not redistributed in this repository, so users should follow the licenses and attribution requirements of the relevant upstream resources.