CoolFace
Datasetpublic

UBC-NLP/SimbaBench_dataset

SibmaBench Data Release & Benchmarking To evaluate your model on SimbaBench across all supported tasks (ASR, TTS, and SLID), simply load the corresponding configuration for the task and language you wish to benchmark. Each task is organized by configuration name (e.g., asr_test_afr, tts_test_wol, slid_61_test). Loading a configuration provides the standardized evaluation split for that specific benchmark. Example: from datasets import load_dataset data =… See the full description on the dataset page: https://huggingface.co/datasets/UBC-NLP/SimbaBench_dataset.

sourceHugging Facecc-by-4.0updated 7mo agoView on Hugging Face
0likes1.9kdownloads
Dataset Card

<div align="center">

<img src="https://africa.dlnlp.ai/simba/images/VoC_logo.png" alt="VoC Logo">

![EMNLP 2025 Paper](https://aclanthology.org/2025.emnlp-main.559/) ![Official Website](https://africa.dlnlp.ai/simba/) ![SimbaBench](https://huggingface.co/spaces/UBC-NLP/SimbaBench) ![GitHub Repository](https://github.com/UBC-NLP/simba) ![Hugging Face](https://huggingface.co/collections/UBC-NLP/simba-speech-series) ![Hugging Face Dataset](https://huggingface.co/datasets/UBC-NLP/SimbaBench_dataset)

</div>

SibmaBench Data Release & Benchmarking

To evaluate your model on SimbaBench across all supported tasks (ASR, TTS, and SLID), simply load the corresponding configuration for the task and language you wish to benchmark.

Each task is organized by configuration name (e.g., asr_test_afr, tts_test_wol, slid_61_test). Loading a configuration provides the standardized evaluation split for that specific benchmark.

Example:

python
from datasets import load_dataset

data = load_dataset("UBC-NLP/SimbaBench_dataset", "asr_test_afr")
DatasetDict({
    test: Dataset({
        features: ['split', 'benchmark_id', 'audio', 'text', 'duration_s', 'lang_iso3', 'lang_name'],
        num_rows: 1000
    })
})
python
data['test'][0]
{'split': 'test',
 'benchmark_id': 'afr_Lwazi_afr_test_idx3889',
 'audio': {'path': None,
  'array': array([ 4.27246094e-04,  7.62939453e-04,  6.71386719e-04, ...,
         -3.05175781e-04, -2.13623047e-04, -6.10351562e-05]),
  'sampling_rate': 16000},
 'text': 'watter, verontwaardiging sou daar, in ons binneste gewees het?',
 'duration_s': 5.119999885559082,
 'lang_iso3': 'afr',
 'lang_name': 'Afrikaans'}

📌 ASR Evaluation Configurations

<details>

Config NameLanguageISO# Samples# Hours
asrtestAkuapim-twiAkuapim-twiAkuapim-twi1,0001.35
asrtestAsante-twiAsante-twiAsante-twi1,0000.97
asrtestafrAfrikaansafr1,0000.87
asrtestamhAmharicamh5811.12
asrtestbasBasaabas5820.76
asrtestbemBembabem1,0002.15
asrtestdavTaitadav8781.17
asrtestdyuDyuladyu590.10
asrtestfatFantifat1,0001.38
asrtestfonFonfon1,0000.66
asrtestfucPulaarfuc1000.10
asrtestfufPularfuf1290.03
asrtestgaaGagaa1,0001.52
asrtesthauHausahau6810.89
asrtestiboIgboibo50.01
asrtestkabKabylekab1,0001.05
asrtestkinKinyarwandakin1,0001.50
asrtestklnKalenjinkln1,0001.50
asrtestlozLoziloz3990.91
asrtestlugGandalug1,0001.65
asrtestluoLuo (Kenya and Tanzania)luo1,0001.31
asrtestmlqWestern Maninkakanmlq1820.04
asrtestnblSouth Ndebelenbl1,0001.12
asrtestnsoNorthern Sothonso1,0000.88
asrtestnyaNyanjanya4281.31
asrtestsotSouthern Sothosot1,0000.82
asrtestsrrSerersrr8992.84
asrtestsswSwatissw1,0000.93
asrtestsusSususus2100.05
asrtestswaSwahiliswa1,0001.23
asrtesttigTigretig1850.33
asrtesttirTigrinyatir70.01
asrtesttoiTonga (Zambia)toi4631.47
asrtesttsnTswanatsn1,0000.82
asrtesttsoTsongatso1,0000.99
asrtesttwiTwitwi120.02
asrtestvenVendaven1,0000.92
asrtestwolWolofwol1,0001.19
asrtestxhoXhosaxho1,0000.92
asrtestyorYorubayor3590.42
asrtestzghStandard Moroccan Tamazightzgh1970.22
asrtestzulZuluzul1,0001.10

</details>

📌 TTS Evaluation Configurations

<details>

Config NameLanguageISO# Samples# Hours
ttstesteweEweewe660.29
ttstestkinKinyarwandakin1,0531.30
ttstestAsante-twiAsante-twiAsante-twi640.18
ttstestyorYorubayor400.13
ttstestwolWolofwol4,0014.12
ttstesthauHausahau1240.24
ttstestlinLingalalin630.28
ttstestxhoXhosaxho2420.31
ttstesttsnTswanatsn2380.36
ttstestafrAfrikaansafr2930.34
ttstestsotSouthern Sothosot2100.33
ttstestAkuapim-twiAkuapim-twiAkuapim-twi830.22

</details>

📌 SLID Evaluation

<details>

Config NameLanguage Scope# Samples# Hours
slid61test61 Languages21,81734.36

</details>

📌 Training data

<details>

Config NameLanguage# Samples (Train / Dev)# Hours (Train / Dev)
asrtraindev42 languages218,515 / 7,500204.51 / 10.49
slidtraindev49 languages355,337 / 8,529330.50 / 13.49
ttstraindev7 languages81,729 / 1,347140.51 / 3.49
TOTAL655,581 / 17,376675.52 / 27.47

</details>


📖 Citation

If you use the SimbaBench dataset for your scientific publication, or if you find the resources in this repository useful, please cite our paper and the original dataset papers.

📄 SimbaBench Paper
bibtex
@inproceedings{elmadany-etal-2025-voice,
    title = "Voice of a Continent: Mapping {A}frica{'}s Speech Technology Frontier",
    author = "Elmadany, AbdelRahim A.  and
      Kwon, Sang Yun  and
      Toyin, Hawau Olamide  and
      Alcoba Inciarte, Alcides  and
      Aldarmaki, Hanan  and
      Abdul-Mageed, Muhammad",
    editor = "Christodoulopoulos, Christos  and
      Chakraborty, Tanmoy  and
      Rose, Carolyn  and
      Peng, Violet",
    booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
    month = nov,
    year = "2025",
    address = "Suzhou, China",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2025.emnlp-main.559/",
    doi = "10.18653/v1/2025.emnlp-main.559",
    pages = "11039--11061",
    ISBN = "979-8-89176-332-6",
}
📄 Original ASR Datasets Citation

<details> <summary><b>Besacier & Gauthier, 2023 — Alffa Public</b></summary>

bibtex
@misc{besacier-gauthier-2023-alffa,
  author       = {Besacier, Laurent and Gauthier, Elodie},
  title        = {{ALFFA\_PUBLIC}: {A}frican Languages Factored Lattices
                  for Automatic Speech Recognition},
  year         = {2023},
  howpublished = {\url{https://github.com/getalp/ALFFA_PUBLIC}},
}

</details>

<details> <summary><b>Sikasote & Anastasopoulos, 2022 — BembaSpeech</b></summary>

bibtex
@inproceedings{sikasote-anastasopoulos-2022-bembaspeech,
  author    = {Sikasote, Claytone and Anastasopoulos, Antonios},
  title     = {{BembaSpeech}: A Speech Recognition Corpus for the
               Bemba Language},
  booktitle = {Proceedings of the Language Resources and Evaluation
               Conference},
  pages     = {7277--7283},
  year      = {2022},
  address   = {Marseille, France},
  publisher = {European Language Resources Association},
}

</details>

<details> <summary><b>Mozilla Foundation, 2023 — Common Voice (CV-19)</b></summary>

bibtex
@misc{mozilla-2023-commonvoice,
  author       = {{Mozilla Foundation}},
  title        = {Mozilla Common Voice: A Massively Multilingual Open
                  Dataset for Voice Technologies},
  year         = {2023},
  howpublished = {\url{https://commonvoice.mozilla.org}},
}

</details>

<details> <summary><b>Asamoah Owusu et al., 2022 — Financial Speech</b></summary>

bibtex
@misc{asamoahowusu-etal-2022-financialspeech,
  author       = {{Asamoah Owusu}, D. and Korsah, A. and Quartey, B.
                  and {Nwolley Jnr.}, S. and Sampah, D.
                  and Adjepon-Yamoah, D. and {Omane Boateng}, L.},
  title        = {Financial Inclusion Speech Dataset},
  year         = {2022},
  howpublished = {\url{https://github.com/Ashesi-Org/Financial-Inclusion-Speech-Dataset}},
  note         = {Created by Ashesi University and Nokwary Technologies
                  with funding from Lacuna Fund},
}

</details>

<details> <summary><b>Gauthier et al., 2024 — Kallaama</b></summary>

bibtex
@inproceedings{gauthier-etal-2024-kallaama,
  author    = {Gauthier, Elodie and Ndiaye, Aminata and Guissé, Abdoulaye},
  title     = {Kallaama: A Transcribed Speech Dataset about Agriculture
               in the Three Most Widely Spoken Languages in {S}enegal},
  booktitle = {Proceedings of the Fifth Workshop on Resources for
               African Indigenous Languages ({RAIL}) @ {LREC-COLING} 2024},
  year      = {2024},
  address   = {Lannion, France; Dakar and Thiès, Sénégal},
}

</details>

<details> <summary><b>Van Heerden et al., 2016 — Lwazi</b></summary>

bibtex
@inproceedings{vanheerden-etal-2016-lwazi,
  author    = {Van Heerden, Charl and Kleynhans, Neil and Davel, Marelie H.},
  title     = {Improving the {Lwazi} {ASR} Baseline},
  booktitle = {Proceedings of Interspeech 2016},
  year      = {2016},
}

</details>

<details> <summary><b>NaijaVoices, 2024 — Naija Voices</b></summary>

bibtex
@misc{naijavoices-2024,
  author       = {{NaijaVoices}},
  title        = {{NaijaVoices} Dataset: A Multilingual Speech Corpus
                  for {N}igerian Languages},
  year         = {2024},
  howpublished = {\url{https://naijavoices.com/}},
}

</details>

<details> <summary><b>Barnard et al., 2014 — NCHLT + AUX1/2</b></summary>

bibtex
@inproceedings{barnard-etal-2014-nchlt,
  author    = {Barnard, Etienne and Davel, Marelie H. and
               Van Heerden, Charl and De Wet, Febe and Badenhorst, Jaco},
  title     = {The {NCHLT} Speech Corpus of the {S}outh {A}frican Languages},
  booktitle = {Proceedings of the 2014 Spoken Language Technologies for
               Under-resourced Languages ({SLTU}) Workshop},
  pages     = {194--200},
  year      = {2014},
  address   = {St. Petersburg, Russia},
}

</details>

<details> <summary><b>Doumbouya et al., 2021 — Nicolingua (0003 & 0004)</b></summary>

bibtex
@inproceedings{doumbouya-etal-2021-nicolingua,
  author    = {Doumbouya, Moussa and Einstein, Lisa and Piech, Chris},
  title     = {Using Radio Archives for Low-Resource Speech Recognition:
               Towards an Intelligent Virtual Assistant for Illiterate Users},
  booktitle = {Proceedings of the {AAAI} Conference on Artificial Intelligence},
  volume    = {35},
  year      = {2021},
}

</details>

<details> <summary><b>Gutkin et al., 2020 — YorubaVoice</b></summary>

bibtex
@inproceedings{gutkin-etal-2020-yoruba,
  author    = {Gutkin, Alexander and Demirşahin, Işın and
               Kjartansson, Oddur and Rivera, Clara and Túbòsún, Kólá},
  title     = {Developing an Open-Source Corpus of {Y}oruba Speech},
  booktitle = {Proceedings of Interspeech 2020},
  pages     = {404--408},
  year      = {2020},
  address   = {Shanghai, China},
  publisher = {International Speech and Communication Association ({ISCA})},
  doi       = {10.21437/Interspeech.2020-1096},
}

</details>

<details> <summary><b>Sikasote et al., 2023 — Zambezi Voice (ASR & Audio Only)</b></summary>

bibtex
@inproceedings{sikasote-etal-2023-zambezi,
  author    = {Sikasote, Claytone and Siaminwe, Kalinda and Mwape, Stanly
               and Zulu, Bangiwe and Phiri, Mofya and Phiri, Martin
               and Zulu, David and Nyirenda, Mayumbo and Anastasopoulos, Antonios},
  title     = {{Zambezi Voice}: A Multilingual Speech Corpus for {Z}ambian Languages},
  booktitle = {Proc. Interspeech 2023},
  pages     = {3984--3988},
  year      = {2023},
}

</details>

<details> <summary><b>Van der Westhuizen & Niesler, 2018 — SO (Code-Switched)</b></summary>

bibtex
@inproceedings{derwesthuizen-niesler-2018-soap,
  author    = {{Van der Westhuizen}, Ewald and Niesler, Thomas},
  title     = {A First {S}outh {A}frican Corpus of Multilingual Code-switched
               Soap Opera Speech},
  booktitle = {Proceedings of the Eleventh International Conference on
               Language Resources and Evaluation ({LREC} 2018)},
  year      = {2018},
  address   = {Miyazaki, Japan},
  publisher = {European Language Resources Association ({ELRA})},
}

</details>

<details> <summary><b>Modipa et al., 2015 — SPCS (Code-Switched)</b></summary>

bibtex
@inproceedings{modipa-etal-2015-spcs,
  author    = {Modipa, T. I. and Davel, M. H. and De Wet, F.},
  title     = {Implications of {S}epedi/{E}nglish Code Switching for {ASR} Systems},
  booktitle = {Proceedings of the Pattern Recognition Association of
               {S}outh {A}frica ({PRASA})},
  pages     = {112--117},
  year      = {2015},
}

</details>

📄 Original SLID Datasets Citation

<details> <summary><b>Doumbouya et al., 2021 — Nicolingua (0003)</b></summary>

bibtex
@inproceedings{doumbouya-etal-2021-nicolingua,
  author    = {Doumbouya, Moussa and Einstein, Lisa and Piech, Chris},
  title     = {Using Radio Archives for Low-Resource Speech Recognition:
               Towards an Intelligent Virtual Assistant for Illiterate Users},
  booktitle = {Proceedings of the {AAAI} Conference on Artificial Intelligence},
  volume    = {35},
  year      = {2021},
}

</details>

<details> <summary><b>The Brick House Cooperative, 2024 — OlongoAfrica</b></summary>

bibtex
@misc{olongoafrica-2024,
  author       = {{The Brick House Cooperative}},
  title        = {{OlongoAfrica} Multilingual Anthology},
  year         = {2024},
  howpublished = {\url{https://lingua.olongoafrica.com/}},
  note         = {A collection of translated and narrated short stories
                  in various African languages, including Edo, Tamazight,
                  Yoruba, Swahili, Hausa, Tiv, Shona, Ibibio, Igbo,
                  and Nigerian Pidgin},
}

</details>

<details> <summary><b>UDHR Audio, 2025 — UDHR</b></summary>

bibtex
@misc{udhr-audio-2025,
  author       = {{Universal Declaration of Human Rights Audio}},
  title        = {Universal Declaration of Human Rights Audio Project},
  year         = {2025},
  howpublished = {\url{https://udhr.audio/}},
  note         = {Audio recordings of the Universal Declaration of Human
                  Rights in multiple languages},
}

</details>

<details> <summary><b>Elmadany et al., 2025 — Voice of Africa (VOA)</b></summary>

bibtex
@inproceedings{elmadany-etal-2025-simba,
  author    = {Elmadany, AbdelRahim and Kwon, Sang Yun and
               Toyin, Hawau Olamide and Inciarte, Alcides Alcoba and
               Aldarmaki, Hanan and Abdul-Mageed, Muhammad},
  title     = {Voice of a Continent: Mapping {A}frica's Speech
               Technology Frontier},
  booktitle = {Proceedings of the 2025 Conference on Empirical Methods
               in Natural Language Processing},
  pages     = {11028--11050},
  year      = {2025},
  publisher = {Association for Computational Linguistics},
  url       = {https://aclanthology.org/2025.emnlp-main.559},
}

</details>

<details> <summary><b>Valk & Alumäe, 2021 — VoxLingua</b></summary>

bibtex
@inproceedings{valk-alumae-2021-voxlingua,
  author    = {Valk, Jörgen and Alumäe, Tanel},
  title     = {{VoxLingua107}: A Dataset for Spoken Language Recognition},
  booktitle = {Proceedings of the {IEEE} Spoken Language Technology
               Workshop ({SLT})},
  year      = {2021},
}

</details>

<details> <summary><b>Sikasote et al., 2023 — Zambezi Voice (Audio Only)</b></summary>

bibtex
@inproceedings{sikasote-etal-2023-zambezi,
  author    = {Sikasote, Claytone and Siaminwe, Kalinda and Mwape, Stanly
               and Zulu, Bangiwe and Phiri, Mofya and Phiri, Martin
               and Zulu, David and Nyirenda, Mayumbo and Anastasopoulos, Antonios},
  title     = {{Zambezi Voice}: A Multilingual Speech Corpus for {Z}ambian Languages},
  booktitle = {Proc. Interspeech 2023},
  pages     = {3984--3988},
  year      = {2023},
}

</details>

📄 Original TTS Datasets Citation

<details> <summary><b>Meyer et al., 2022 — BibleTTS</b></summary>

bibtex
@inproceedings{meyer-etal-2022-bibletts,
  author    = {Meyer, Josh and Adelani, David and Casanova, Edresson
               and Öktem, Alp and Whitenack, Daniel and Weber, Julian
               and {Kabongo Kabenamualu}, Salomon and Salesky, Elizabeth
               and Orife, Iroro and Leong, Colin and Ogayo, Perez
               and {Chinenye Emezue}, Chris and Mukiibi, Jonathan
               and Osei, Salomey and Agbolo, Apelete and Akinode, Victor
               and Opoku, Bernard and Samuel, Olanrewaju and Alabi, Jesujoba
               and Muhammad, Shamsuddeen Hassan},
  title     = {{BibleTTS}: A Large, High-Fidelity, Multilingual, and
               Uniquely {A}frican Speech Corpus},
  booktitle = {Interspeech 2022},
  pages     = {2383--2387},
  year      = {2022},
  address   = {Incheon, Korea},
  publisher = {ISCA},
  doi       = {10.21437/Interspeech.2022-10850},
}

</details>

<details> <summary><b>van Niekerk et al., 2017 — High-Quality TTS (SA)</b></summary>

bibtex
@inproceedings{vanniekerk-etal-2017-hqtts,
  author    = {van Niekerk, Daniel and van Heerden, Charl and Davel, Marelie
               and Kleynhans, Neil and Kjartansson, Oddur and Jansche, Martin
               and Ha, Linne},
  title     = {Rapid Development of {TTS} Corpora for Four {S}outh {A}frican Languages},
  booktitle = {Interspeech 2017},
  pages     = {2178--2182},
  year      = {2017},
  address   = {Stockholm, Sweden},
  publisher = {ISCA},
  doi       = {10.21437/Interspeech.2017-1139},
}

</details>

<details> <summary><b>Digital Umuganda, 2023 — Kinyarwanda TTS</b></summary>

bibtex
@misc{digitalumuganda-2023-kinyarwanda,
  author       = {{Digital Umuganda}},
  title        = {{AfriSpeech} {K}inyarwanda Male and Female {TTS} Datasets},
  year         = {2023},
  howpublished = {\url{https://huggingface.co/datasets/DigitalUmuganda/afrispeak_kinyarwanda_male_tts_dataset}},
}

</details>