masakhane
mafandMAFAND-MT is the largest MT benchmark for African languages in the news domain, covering 21 languages. The languages covered are:
- Amharic
- Bambara
- Ghomala
- Ewe
- Fon
- Hausa
- Igbo
- Kinyarwanda
- Luganda
- Luo
- Mossi
- Nigerian-Pidgin
- Chichewa
- Shona
- Swahili
- Setswana
- Twi
- Wolof
- Xhosa
- Yoruba
- Zulu
The train/validation/test sets are available for 16 languages, and validation/test set for amh, kin, nya, sna, and xho
For more details see https://aclanthology.org/2022.naacl-main.223/afrimgsm
Dataset Card for afrimgsm
Dataset Summary
AFRIMGSM is an evaluation dataset comprising translations of a subset of the GSM8k dataset into 16 African languages.
It includes test sets across all 18 languages, maintaining an English and French subsets from the original GSM8k dataset.
Languages
There are 18 languages available :
Dataset Structure
Data Instances
The examples look like this for English:
from datasets import load_dataset
data =… See the full description on the dataset page: https://huggingface.co/datasets/masakhane/afrimgsm.masakhanews
Dataset Card for [Dataset Name]
Dataset Summary
MasakhaNEWS is the largest publicly available dataset for news topic classification in 16 languages widely spoken in Africa.
The train/validation/test sets are available for all the 16 languages.
Supported Tasks and Leaderboards
[More Information Needed]
news topic classification: categorize news articles into new topics e.g business, sport sor politics.
Languages
There are 16 languages available :… See the full description on the dataset page: https://huggingface.co/datasets/masakhane/masakhanews.afrimmlu
Dataset Card for afrimmlu
Dataset Summary
AFRIMMLU is an evaluation dataset comprising translations of a subset of the MMLU dataset into 15 African languages.
It includes test sets across all 17 languages, maintaining an English and French subsets from the original MMLU dataset.
Languages
There are 17 languages available :
Dataset Structure
Data Instances
The examples look like this for English:
from datasets import load_dataset
data =… See the full description on the dataset page: https://huggingface.co/datasets/masakhane/afrimmlu.afrisenti
Dataset Summary
AfriSenti is the largest sentiment analysis dataset for under-represented African languages, covering 110,000+ annotated tweets in 14 African languages (Amharic, Algerian Arabic, Hausa, Igbo, Kinyarwanda, Moroccan Arabic, Mozambican Portuguese, Nigerian Pidgin, Oromo, Swahili, Tigrinya, Twi, Xitsonga, and Yoruba).
The datasets are used in the first Afrocentric SemEval shared task, SemEval 2023 Task 12: Sentiment analysis for African languages (AfriSenti-SemEval).… See the full description on the dataset page: https://huggingface.co/datasets/masakhane/afrisenti.afrixnli
Dataset Card for afrixnli
Dataset Summary
AFRIXNLI is an evaluation dataset comprising translations of a subset of the XNLI dataset into 16 African languages.
It includes both validation and test sets across all 18 languages, maintaining the English and French subsets from the original XNLI dataset.
Languages
There are 18 languages available :
Dataset Structure
Data Instances
The examples look like this for English:
from datasets import… See the full description on the dataset page: https://huggingface.co/datasets/masakhane/afrixnli.