SaiedAlshahrani/Detect-Egyptian-Wikipedia-Articles

<center><h1> Detect Egyptian Wikipedia <i>Template-translated</i> Articles </h1></center>

Dataset Description:

We release the heuristically filtered, manually processed, and automatically classified Egyptian Arabic Wikipedia articles dataset. This dataset was used to develop a web-based detection system to automatically identify the template-translated articles on the Egyptian Arabic Wikipedia edition. The system is called **Egyptian Arabic Wikipedia Scanner** and is hosted on Hugging Face Spaces, here: **SaiedAlshahrani/Detect-Egyptian-Wikipedia-Articles**.

This dataset is introduced in a research paper titled "***Leveraging Corpus Metadata to Detect Template-based Translation: An Exploratory Case Study of the Egyptian Arabic Wikipedia Edition***", which is accepted at LREC-COLING 2024: The 6th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT6), and is currently released under an MIT license.

Dataset Sources:

This Egyptian Arabic Wikipedia articles dataset was extracted from the complete Wikipedia dumps of the Egyptian Arabic Wikipedia edition, downloaded on the 1st of January 2024, and processed using the Gensim Python library.

Dataset Features:

We utilized the Wikimedia XTools API to collect the metadata (dataset features) of the Egyptian Arabic Wikipedia articles. Specifically, we collected the following metadata/features for each article: total edits, total editors, top editors, total bytes, total characters, total words, creator name, and creation date.

Dataset Subsets:

1. 1.Balanced: A balanced subset of the dataset comprised 20K (10K for each class) and was split in the ratio of 80:20 for training and testing. This subset was filtered and processed using selected heuristic rules.
2. 2.Unbalanced: An unbalanced subset of the dataset comprised 166K and was split in the ratio of 80:20 for training and testing. This subset is the rest of the filtered and processed articles using selected heuristic rules.
3. 3.Uncategorized: Another unbalanced subset of the dataset comprised 569K and was split in the ratio of 80:20 for training and testing, but this was classified automatically using the XGBoost classifier trained using the balanced subset.

Dataset Citations:

<s>Saied Alshahrani, Hesham Haroon, Ali Elfilali, Mariama Njie, and Jeanna Matthews. 2024. Leveraging Corpus Metadata to Detect Template-based Translation: An Exploratory Case Study of the Egyptian Arabic Wikipedia Edition. arXiv preprint arXiv:2404.00565.</s>

Saied Alshahrani, Hesham Haroon, Ali Elfilali, Mariama Njie, and Jeanna Matthews. 2024. Leveraging Corpus Metadata to Detect Template-based Translation: An Exploratory Case Study of the Egyptian Arabic Wikipedia Edition. In Proceedings of the 6th Workshop on Open-Source Arabic Corpora and Processing Tools (OSACT) with Shared Tasks on Arabic LLMs Hallucination and Dialect to MSA Machine Translation @ LREC-COLING 2024, pages 31–45, Torino, Italia. ELRA and ICCL.*