johneze/chichewa-text2sql

Chichewa Text-to-SQL

The first structured Text-to-SQL benchmark for Chichewa, a low-resource Bantu language spoken by over 12 million people in Malawi and neighboring regions. The dataset contains 400 manually curated natural language–SQL pairs in both Chichewa (Nyanja) and English, grounded in a unified relational SQLite database covering five real-world domains from Malawi.

Dataset Summary

This benchmark was constructed to investigate the adaptation of Large Language Models (LLMs) for Text-to-SQL generation in Chichewa. It supports systematic evaluation of zero-shot, few-shot (random and retrieval-augmented), and parameter-efficient fine-tuning (QLoRA) approaches for low-resource semantic parsing.

Key findings from the accompanying research:

• —English zero-shot execution accuracy: 20% → 50% (random few-shot) → 70% (RAG few-shot) → 76.7% (QLoRA)
• —Chichewa zero-shot execution accuracy: 0% across all models → 41.7% (RAG few-shot) → 41.7% (QLoRA) → 53.3% (QLoRA + RAG few-shot)

Database Schema

The SQLite database (database/chichewa_text2sql.db) contains five tables:

Dataset Structure

Files

Data Fields

Each example contains:

{
  "id": 1,
  "question_en": "Which district produced the most Maize",
  "question_ny": "Ndi boma liti komwe anakolola chimanga chambiri?",
  "sql_statement": "SELECT district, MAX(yield) AS max_yield FROM production WHERE crop = 'Maize';",
  "sql_result": "[('Lilongwe', 444440.0)]",
  "difficulty_level": "easy",
  "table": "production"
}

Difficulty Levels

• —easy — single table, simple SELECT / WHERE / ORDER BY
• —medium — aggregations, GROUP BY, LIMIT
• —hard — multi-condition queries, subqueries, JOINs

Splits

Usage

import json

with open("data/train.json") as f:
    train = json.load(f)

print(train[0]["question_ny"])    # Chichewa question
print(train[0]["question_en"])    # English question
print(train[0]["sql_statement"])  # Ground-truth SQL

Citation

If you use this dataset, please cite:

@dataset{eze2026chichewa,
  author    = {Eze, John Emeka and Matekenya, Dunstan and Matthewe, Evance},
  title     = {Bridging the Language Gap in Text-to-SQL: Adapting LLMs for Chichewa in a Low-Resource Setting},
  year      = {2026},
  publisher = {HuggingFace},
  url       = {https://huggingface.co/datasets/johneze/chichewa-text2sql}
}

License

MIT