Fatika01/nigeria-livestock-llm-benchmark
Nigeria Livestock LLM Benchmark Dataset Description A 420-question benchmark evaluating LLM performance on Nigerian livestock management knowledge, designed to investigate geographic and cultural bias in large language models. Why This Matters Most LLM benchmarks are Western-centric. This dataset targets a critical gap: how well do frontier models perform on domain knowledge specific to sub-Saharan African agricultural contexts?… See the full description on the dataset page: https://huggingface.co/datasets/Fatika01/nigeria-livestock-llm-benchmark.
Nigeria Livestock LLM Benchmark
Dataset Description
A 420-question benchmark evaluating LLM performance on Nigerian livestock management knowledge, designed to investigate geographic and cultural bias in large language models.
Why This Matters
Most LLM benchmarks are Western-centric. This dataset targets a critical gap: how well do frontier models perform on domain knowledge specific to sub-Saharan African agricultural contexts?
Dataset Structure
- 420 multiple-choice questions
- Categories include: breeds, reproduction, nutrition, disease, traditional management practices
- Questions sourced from Nigerian livestock management literature
Models Evaluated
Preliminary Findings
Llama 3.1 8B pilot shows a bias gradient across knowledge categories, with performance dropping significantly on traditional and indigenous livestock management questions.
Intended Use
- LLM bias research
- AI safety evaluation in Global South contexts
- Agricultural AI benchmarking
Citation
If you use this dataset, please cite this repository and link to: GitHub Live Demo
Author
Fatika — Veterinary student & ML researcher, Nigeria HuggingFace: Fatika01