DatarrX/myanmar-Wikipedia

Myanmar Wikipedia Dataset (20260501)

This dataset contains a cleaned, processed, and high-quality collection of Burmese Wikipedia articles, curated to serve as a robust foundation for Natural Language Processing (NLP) and Artificial Intelligence development in the Burmese language.

🏛️ About DatarrX

DatarrX (Burmese: ဒေတာအက်စ်) is a non-profit open-source foundation dedicated to building a robust digital foundation for the Burmese language in the AI era. We believe that high-quality data is the core of AI innovation. Our mission is to empower the Burmese AI ecosystem by building high-quality open-source datasets, essential AI tools, and localized technical resources.

🏗️ Dataset Creation Pipeline

This dataset was developed through a rigorous multi-stage cleaning and preparation process to ensure maximum utility for researchers and developers:

1. 1.Data Extraction: The source material was derived from the official Burmese Wikipedia Database Dump (2026-05-01).
2. 2.Markup Sanitization: We systematically removed MediaWiki markup, HTML tags, template artifacts ({{...}}), and internal links ([[...]]).
3. 3.Filtering & Noise Reduction: * Applied strict Zawgyi-to-Unicode verification to ensure data consistency.
4. 4.Removed tabular metadata, non-informative CSS/module snippets, and malformed rows.
5. 5.Filtered out rows containing non-Burmese characters (where applicable) and low-information symbols.
6. 6.Sentence Segmentation & Normalization: * Articles were split into individual sentences.
7. 7.We enforced a standard Burmese full-stop (။) at the end of every sentence to ensure structural consistency.
8. 8.Syllable Tokenization: Using mm-syllable, every sentence was segmented into its constituent Burmese syllables, providing a secondary column (syllable) for syllable-level model training.

📊 Dataset Statistics

• —Total Sentences: 546,045
• —Average Sentence Length: 122.98 characters
• —Average Syllable Count: 39.11
• —Structural Consistency: 99.90% of sentences end with the standard Burmese full-stop (။).

Data Structure

Sample Entry:

{
  "text": "အကြောင်းအရာပေါင်း ခုကို ဖြင့် ဖတ်ရှုနိုင်ပါသည်။",
  "syllable": "အ ကြောင်း အ ရာ ပေါင်း ခု ကို ဖြင့် ဖတ် ရှု နိုင် ပါ သည် ။",
  "text_len": 47,
  "syllable_count": 14
}

👨‍💻 Creator & Contributors

This dataset is curated and maintained by Khant Sint Heinn (Burmese: ခန့်ဆင့်ဟိဏ်း, Kalix Louis).

Khant Sint Heinn is a Machine Learning Engineer specializing in NLP and open-source AI development. As the Lead Developer at DatarrX, he focuses on building scalable data pipelines and practical infrastructure to improve support for low-resource languages. His work aims to transform limited language resources into practical opportunities through clean data and community-driven innovation.

⚖️ License & Citation

This work is released under the Creative Commons Attribution 4.0 International (CC-BY-4.0) license, ensuring openness and collaboration for the global AI research community.

If you find this dataset useful for your research or applications, please cite it as:

@misc{datarrx_my_wiki_2026,
  author       = {Khant Sint Heinn},
  title        = {yanmar Wikipedia Dataset (20260501)},
  year         = {2026},
  publisher    = {Hugging Face},
  organization = {DatarrX},
  howpublished = {https://huggingface.co/datasets/DatarrX/myanmar-Wikipedia},
  note         = {Published under DatarrX. Open-source community asset released under Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)}
}