alakxender/dhivehi-transliteration-pairs

Dhivehi Transliteration Pairs

This dataset contains 187,908 aligned sentence pairs in Dhivehi (Thaana script) and its romanized (transliterated) Latin script form, making it a valuable resource for machine translation, cross-lingual NLP research, and bilingual corpus analysis.

Dataset Details

• —Language pair: English ↔ Dhivehi (Thaana script)
• —Train examples: 150,326
• —Test examples: 37,582
• —Total examples: 187,908

Dataset Structure

DatasetDict({
    train: Dataset({
        features: ['latin', 'dh', 'src'],
        num_rows: 150,326
    }),
    test: Dataset({
        features: ['latin', 'dh', 'src'],
        num_rows: 37,582
    })
})

Language Distribution

Length Distribution

Source Distribution

The dataset contains content from 6 different sources.

Note: This dataset’s accuracy has been assessed using a hybrid approach that includes automatic transliteration checks, semantic similarity for low-confidence cases, and a small amount of manual review. Additional analysis can be performed to confirm it meets specific needs.

Dataset Format

Each entry contains three fields:

{
    "latin": "Bihloorigandu thalhaalumugai thuhumathu kurevey meehun dhookohlaifi", 
    "dh": "ބިއްލޫރިގަނޑު ތަޅާލުމުގައި ތުހުމަތު ކުރެވޭ މީހުން ދޫކޮށްލައިފި", 
    "src": "617661736d76"
}

Usage

from datasets import load_dataset

dataset = load_dataset("alakxender/dhivehi-transliteration-pairs")

# Access train and test splits
train_data = dataset['train']
test_data = dataset['test']

# Example entry
print(train_data[0])
# {'latin': 'latin text', 'dh': 'ދިވެހި ލިޔުން', 'src': 'source_id'}

Fields

• —latin: English text
• —dh: Dhivehi text in Thaana script
• —src: Source identifier

Use Cases

• —Machine Translation: Latin ↔ Dhivehi translation systems
• —Cross-lingual NLP: Multilingual embeddings, cross-lingual classification
• —Linguistic Research: Dhivehi language analysis, translation studies
• —Educational Tools: Language learning applications

Data Sources

This dataset combines content from multiple sources. The src field indicates the origin of each entry, enabling source tracing and source-specific analysis. The src value is stored as a hexadecimal string, which can be decoded to identify the source name. Data quality may vary depending on the src value.

The data in this dataset is collected from publicly available sources. All content remains the property of its respective owners, and credit is due to the original creators and publishers. This dataset is provided for research and educational purposes, with the intention of supporting work in Dhivehi language processing and related fields.