Okwu/african-language-parallel-corpus
African Language Parallel Corpus Human-created, human-validated parallel sentence pairs for three African languages, released openly by Okwu. Version 1.0. Dataset summary A parallel corpus of everyday-register sentence pairs for Yorùbá, Swahili, and Nigerian Pidgin, each paired with English. The core is derived from NKENNE's own language-learning curriculum — content authored and reviewed by native-speaker educators — supplemented for Swahili with public-domain… See the full description on the dataset page: https://huggingface.co/datasets/Okwu/african-language-parallel-corpus.
African Language Parallel Corpus
Human-created, human-validated parallel sentence pairs for three African languages, released openly by Okwu. Version 1.0.
Dataset summary
A parallel corpus of everyday-register sentence pairs for Yorùbá, Swahili, and Nigerian Pidgin, each paired with English. The core is derived from NKENNE's own language-learning curriculum — content authored and reviewed by native-speaker educators — supplemented for Swahili with public-domain material. Every pair has been validated by a human reviewer checking that source and translation correspond in both directions.
Final counts are frozen at release and match the release manifest and citation exactly.
Languages
Yorùbá (yor), Swahili (swa), Nigerian Pidgin (pcm), each paired with English (eng). All text is Unicode NFC-normalised. Yorùbá combines a dot-below with a tone mark on the same vowel; some combinations have no single precomposed codepoint, so consumers comparing strings should normalise to NFC.
Dataset structure
Each record:
Metadata scope. The schema supports richer fields (dialect, domain, register, quality tier, provenance category, reviewer role, error tags, lifecycle status). These are schema-supported but not populated in v1.0 — the v1.0 records carry text, provenance, and a binary validation flag. Populating the richer fields is planned for subsequent versions.
Curation
Source. Core content is drawn from NKENNE's language-learning curriculum, authored and reviewed by native-speaker educators. Swahili is supplemented with Voice of America material (U.S. federal government, public domain), filtered to VOA-service content only — records carrying AP/Reuters/AFP wire attribution were removed, and every VOA record retains its source_url for audit. No third-party copyrighted text is included; Yorùbá and Pidgin are entirely owned lesson content.
Validation. Each pair was reviewed by a qualified human validator who confirmed that the source sentence and its English translation correspond in both directions. Records that failed validation are not included.
Every release additionally passes automated checks before publication: schema conformance, NFC consistency, and duplicate detection against the existing corpus. 137 submitted rows were excluded at ingest for v1.1 — sentences broken mid-way by an upstream splitting error, rows left untranslated, and duplicates.
Considerations for use
Register. Content is everyday/beginner register (lesson-derived): greetings, conversation, common vocabulary. It is well-suited to foundational MT and language-learning applications and is not a broad-domain or literary corpus.
Composition. Includes short phrases and single-word vocabulary entries alongside full sentences. Some source text carries minor transcription artifacts typical of curriculum content.
Translation depth (Nigerian Pidgin). Pidgin is English-lexified, so high word overlap with the source is expected. Even allowing for that, roughly 340 records added in v1.1 sit close to the English — carrying Pidgin function words and orthography over otherwise English phrasing. They are genuine renderings rather than untranslated text, but they are thin, and they cluster in one contributed batch.
Diacritics (Yorùbá). A small number of Yorùbá records omit the sub-dot characters ẹ, ọ and ṣ entirely. These are distinct letters in Yorùbá orthography rather than optional accents, so their absence can be ambiguous. They are concentrated in a single contributed batch. Tone marks are not separately validated.
Register and metadata are documented rather than hidden — this is a clean, owned, validated seed corpus intended to grow through versioned releases, not a large web-scraped corpus.
Licensing
Data: CDLA-Permissive-2.0. Code and tooling (separate repository): Apache-2.0. Documentation: CC BY 4.0. Attribution required — see citation.
Citation
@dataset{okwu_african_parallel_corpus_2026,
title = {African Language Parallel Corpus},
author = {{Okwu}},
year = {2026},
version= {1.0.0},
url = {https://huggingface.co/datasets/Okwu/african-language-parallel-corpus},
note = {https://github.com/Okwu-AI/corpus}
}