regnant-io/swahili-instruction-22k

Swahili Instruction-Following Dataset (22.5K)

Dataset Description

This dataset contains 22,500 high-quality instruction-following examples in Swahili (Kiswahili), designed for supervised fine-tuning (SFT) of language models. The data was translated from English instruction datasets using the state-of-the-art NLLB-200 translation model and filtered for quality.

Dataset Summary

• —Language: Swahili (sw) - translated from English (en)
• —Size: 22,500 examples
• —Format: JSONL (JSON Lines)
• —Task: Instruction-following / Supervised Fine-Tuning
• —License: Apache 2.0 (inherited from source datasets)

Supported Tasks

• —Instruction Following: Training models to follow natural language instructions
• —Question Answering: Direct question-answer pairs
• —Text Generation: Various generation tasks (summarization, rewriting, creative writing)
• —Conversational AI: Building Swahili-capable chat assistants

Dataset Structure

Data Format

Each example follows the Alpaca instruction format:

{
  "id": 0,
  "instruction": "Toa vidokezo vitatu vya kudumisha afya njema.",
  "input": "",
  "output": "1. Kula lishe bora inayojumuisha matunda na mboga za majani kwa wingi.\n2. Fanya mazoezi mara kwa mara ili kuufanya mwili wako kuwa hodari na wenye nguvu.\n3. Pata usingizi wa kutosha na dumisha ratiba thabiti ya kulala.",
  "source": "alpaca"
}

Data Fields

• —id: Unique identifier for each example
• —instruction: The task instruction in Swahili (what the model should do)
• —input: Optional context or input for the instruction (often empty)
• —output: Expected response in Swahili
• —source: Origin dataset (alpaca, dolly, or flan)

Data Instances

Example 1: Simple instruction without context

{
  "id": 4,
  "instruction": "Toa ufafanuzi wa neno 'usanifu wa programu'.",
  "input": "",
  "output": "Usanifu wa programu ni muundo wa jumla wa mfumo wa kompyuta, unaoonyesha jinsi vipengele vyake mbalimbali vinavyoshirikiana kufanikisha kazi zilizokusudiwa.",
  "source": "alpaca"
}

Example 2: Instruction with input context

{
  "id": 2,
  "instruction": "Badilisha sentensi hii iwe katika hali ya kupita.",
  "input": "Mvulana alimshinda mbwa mbio.",
  "output": "Mbwa alishindwa mbio na mvulana.",
  "source": "alpaca"
}

Dataset Statistics

Source Distribution

The dataset is derived from three high-quality English instruction datasets:

• —Stanford Alpaca: General instruction-following tasks
• —Databricks Dolly-15k: Diverse real-world instructions
• —FLAN Collection: Academic task-oriented instructions

Dataset Creation

Translation Process

1. 1.Source Data Collection: Curated 50,000+ English instruction examples from Alpaca, Dolly-15k, and FLAN
2. 2.Translation: Used facebook/nllb-200-distilled-600M model for English→Swahili translation
3. 3.Quality Filtering:
4. 4.Removed examples with empty outputs
5. 5.Filtered translations with extreme length ratios (< 0.4 or > 3.0)
6. 6.Dropped malformed JSON entries
7. 7.Manual Verification: Spot-checked translations for fluency and accuracy

Translation Model

• —Model: facebook/nllb-200-distilled-600M
• —Method: Beam search (num_beams=4)
• —Max Length: 512 input tokens, 400 generated tokens
• —Language Codes: eng_Latn → swh_Latn

Quality Considerations

Strengths:

• —NLLB-200 provides strong Swahili translation quality (Swahili is well-represented in its training data)
• —Diverse task coverage (QA, summarization, creative writing, technical explanations)
• —Quality filters ensure basic translation sanity

Limitations:

• —Machine-translated data may contain:
• —Slightly formal/literal phrasing compared to natural Swahili
• —Occasional grammar inconsistencies
• —Cultural context mismatches from English sources
• —Not reviewed by native Swahili speakers at scale
• —May contain residual translation artifacts

Recommendations:

• —For production use, consider human post-editing a sample (~2-3K examples) for critical applications
• —Use deduplication post-translation (MT can collapse distinct inputs)
• —Combine with human-written Swahili data if available

Usage

Loading the Dataset

from datasets import load_dataset

dataset = load_dataset("regnant-io/swahili-instruction-22k")

Training Example (Hugging Face)

from transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments
from trl import SFTTrainer

model = AutoModelForCausalLM.from_pretrained("regnant-io/kw5-lite-base")
tokenizer = AutoTokenizer.from_pretrained("regnant-io/kw5-lite-base")

def format_instruction(example):
    if example["input"]:
        return f"### Maelekezo: {example['instruction']}\n\n### Ingizo: {example['input']}\n\n### Jibu: {example['output']}"
    else:
        return f"### Maelekezo: {example['instruction']}\n\n### Jibu: {example['output']}"

trainer = SFTTrainer(
    model=model,
    train_dataset=dataset["train"],
    formatting_func=format_instruction,
    max_seq_length=2048,
)

trainer.train()

Prompt Format

Without Input:

### Maelekezo: [instruction in Swahili]

### Jibu: [response in Swahili]

With Input:

### Maelekezo: [instruction in Swahili]

### Ingizo: [context in Swahili]

### Jibu: [response in Swahili]

Citation

If you use this dataset, please cite the source datasets:

Stanford Alpaca

@misc{alpaca,
  author = {Rohan Taori and Ishaan Gulrajani and Tianyi Zhang and Yann Dubois and Xuechen Li and Carlos Guestrin and Percy Liang and Tatsunori B. Hashimoto},
  title = {Stanford Alpaca: An Instruction-following LLaMA model},
  year = {2023},
  url = {https://github.com/tatsu-lab/stanford_alpaca}
}

Dolly-15k

@misc{dolly15k,
  author = {Databricks},
  title = {databricks-dolly-15k},
  year = {2023},
  url = {https://huggingface.co/datasets/databricks/databricks-dolly-15k}
}

NLLB-200

@article{nllb2022,
  title={No Language Left Behind: Scaling Human-Centered Machine Translation},
  author={NLLB Team and others},
  journal={arXiv preprint arXiv:2207.04672},
  year={2022}
}

Dataset Card Authors

Dataset translated and curated in 2024 for advancing Swahili language AI capabilities.

Licensing Information

Apache 2.0 - inherited from source datasets (Alpaca, Dolly-15k, FLAN)

Ethical Considerations

Intended Use

• —Training small-to-medium Swahili language models (109M - 7B parameters)
• —Research in low-resource language AI
• —Educational applications
• —Swahili chatbot development

Out-of-Scope Use

• —Should not be sole source for high-stakes applications (medical, legal) without human review
• —Not suitable for cultural/regional dialect modeling without additional filtering
• —May not capture contemporary Swahili internet slang or regional variations

Bias and Fairness

• —Source data is English-centric; cultural context may not fully translate
• —NLLB model biases may propagate into translations
• —Limited coverage of East African regional topics
• —Recommend augmentation with locally-sourced Swahili content

Additional Information

Dataset Version

• —Version: 1.0
• —Release Date: 2024
• —Last Updated: 2024

Contact

For questions, issues, or contributions, please open an issue on the dataset repository.

Acknowledgments

• —Meta AI for NLLB-200 translation models
• —Stanford for Alpaca dataset
• —Databricks for Dolly-15k dataset
• —Google for FLAN collection
• —The broader Swahili NLP research community