CoolFace
Datasetpublic

chuuhtetnaing/myanmar-wikipedia-dataset

Myanmar Wikipedia Dataset (Last Crawl Date: 25/03/2025) A collection of scraped Myanmar Wikipedia pages organized by category paths. Overview This dataset contains Myanmar Wikipedia articles scraped based on categorical organization. Unlike the official Wikimedia dataset (subset: 20231101.my), this repository provides an alternative approach to Myanmar Wikipedia content by following the categorical structure starting from the main entry page. Figure 1: The… See the full description on the dataset page: https://huggingface.co/datasets/chuuhtetnaing/myanmar-wikipedia-dataset.

sourceHugging Facecc-by-sa-3.0updated 2y agoView on Hugging Face
1likes18downloads
Dataset Card

Myanmar Wikipedia Dataset (Last Crawl Date: 25/03/2025)

A collection of scraped Myanmar Wikipedia pages organized by category paths.

Overview

This dataset contains Myanmar Wikipedia articles scraped based on categorical organization. Unlike the official Wikimedia dataset (subset: 20231101.my), this repository provides an alternative approach to Myanmar Wikipedia content by following the categorical structure starting from the main entry page.

Category Page Figure 1: The initial category page showing both direct article links and category links

Dataset Structure

Columns

ColumnDescription
titleTitle of the Wikipedia article
contentFull text content of the article
categoriesAll category tags associated with the article (as returned by the API)
urlUnique URL of the Wikipedia article
last_modifiedTimestamp of when the article was last modified
category_pathComplete hierarchical path of categories leading to the article
category_path_countNumber of category paths for the article
unique_wordsList of individual segmented words from title and content
unique_phrasesList of segmented phrases from title and content
unique_mm_wordsList of individual segmented words from title and content (only myanmar chars with some special characters)
unique_mm_phrasesList of segmented phrases from title and content (only myanmar chars with some special characters)

How to Use

You can easily load this dataset using the Hugging Face datasets library:

python
from datasets import load_dataset

ds = load_dataset("chuuhtetnaing/myanmar-wikipedia-dataset")

Data Organization Methodology

This dataset was created with a focus on comprehensive categorical coverage while minimizing redundancy:

  • —Articles are crawled following different category paths
  • —To reduce duplication, identical categories under different paths are excluded
  • —Example: If articles (page-1, page-2, page-3) are found under both:
  • —category_path_1: category-a > category-b > category-c
  • —category_path_2: category-d > category-e > category-c
  • —The second path is excluded to prevent duplication
  • —However, identical articles found under different category names are included:
  • —Example: If articles (page-1, page-2, page-3) can be found under both:
  • —category_path_1: category-a > category-b > category-c
  • —category_path_2: category-a > category-e > category-f
  • —Both paths are included since they represent different categorical organizations

Use Cases

This dataset is particularly useful for:

  • —Research on Myanmar language NLP tasks
  • —Building Myanmar-language models that benefit from categorical understanding
  • —Topic modeling of Myanmar-language content

License

This dataset is free to use for any purpose - personal, commercial, or educational.

Contributions

Feel free to create a pull request if you find any errors or have improvements to suggest for the dataset. Contributions to enhance the accuracy and completeness of the data are welcome.

Acknowledgments

Special thanks to the Wikipedia contributors who created and maintain the original Myanmar Wikipedia content that made this dataset possible.