sk-community/romanized_hindi

Romanized Hindi Dataset

Dataset Description

The Romanized Hindi Dataset is a collection of Hindi text paired with its Romanized (Latin script) representation. It has been created by combining multiple sources, including open datasets, synthetic generation, and rule-based transliteration methods. The dataset is designed for training and evaluating Hindi↔Roman transliteration models.

• —Language(s): Hindi, Romanized Hindi
• —Size: ~1.82M rows
• —License: MIT (check with source datasets for individual terms)

Dataset Sources

1. Dakshina Dataset

• —10,000 rows of Hindi–Romanized pairs.
• —Source: Google Research Dakshina Dataset
• —File: dakshina/dakshina_hi_romanized.csv

2. Synthetic Data

• —150,000 rows of synthetic Hindi↔Roman pairs.
• —Generated using Gemini-2.0-flash and Gemini-2.0-flash-lite.
• —File: synthetic/synthetic_data.csv

3. Converted from CFILT IITB English–Hindi Dataset

• —1.66M rows of Hindi text converted into Romanized Hindi.
• —Source dataset: CFILT IITB English–Hindi
• —Conversion tool: Indic-trans-v2
• —File: opensource/cfilt-iitb-english-hindi.csv

4. Other Open Contributions

• —~14,919 rows collected from open-source transliteration contributions, e.g. crowd-indic-transliteration-data.

Dataset Structure

• —Columns:

• —hi: Original Hindi text (Devanagari script)
• —hi_rom: Romanized Hindi text

• —File format: CSV

Intended Uses

• —Training Hindi→Roman and Roman→Hindi transliteration models
• —Benchmarking transliteration quality (CER, WER, chrF)
• —Research on multilingual and cross-script NLP

Limitations

• —Romanization schemes may vary slightly across sources (Dakshina vs synthetic vs IITB conversion).
• —Synthetic data is machine-generated and may contain occasional noise.
• —The dataset does not guarantee coverage of all dialectal or orthographic variations.

Citation

If you use this dataset, please cite the source:

@article{gharami2025indotranslit,
  title={Modeling Romanized Hindi and Bengali: Dataset Creation and Multilingual LLM Integration},
  author={Kanchon Gharami and Quazi Sarwar Muhtaseem and Deepti Gupta and Lavanya Elluri and Shafika Showkat Moni},
  year={2025}
}

License

This dataset is distributed under the MIT License. Please also check the licenses of the individual source datasets before use.