MaatAI/african-history-sft-synthetic

African History SFT (Chat)

A deduplicated, chat-format supervised fine-tuning (SFT) dataset about African history and culture, assembled from five source datasets and prepared as a ready-to-train train/test split.

Each row is a multi-turn conversation in the standard messages format (system / user / assistant), making it directly usable with tokenizer.apply_chat_template and TRL's SFTTrainer.

Dataset at a glance

• —Split ratio: 95% / 5%, shuffled reproducibly (seed 42).
• —Splits are disjoint — partitioned by row, no leakage between train and test.
• —Languages: primarily English, with a French subset (colonial history).
• —Modality: text only.

Schema

A typical row:

{
  "messages": [
    {"role": "system", "content": "You are an expert on African history..."},
    {"role": "user", "content": "Describe the governance structures of the Asante kingdom."},
    {"role": "assistant", "content": "The Asante kingdom utilized councils such as..."}
  ],
  "source_dataset": "Svngoku/african-history-extra-01-01-26-v1-chat",
  "message_count": 3
}

Source composition

Built from `Svngoku/african-history-merged-v1`.

How it was prepared

1. 1.Pulled all rows from the merged source via DuckDB over the hf:// protocol.
2. 2.Shuffled reproducibly using a stable hash of content_hash (seed 42).
3. 3.Split 95/5 into train/test with datasets.train_test_split (disjoint).
4. 4.Pushed both splits atomically as a DatasetDict.

Usage

from datasets import load_dataset

ds = load_dataset("Svngoku/african-history-sft")
print(ds)
# DatasetDict({train: 25552 rows, test: 1345 rows})

# Format for SFT with any chat model's template
from transformers import AutoTokenizer
tok = AutoTokenizer.from_pretrained("unsloth/Qwen3-4B-Instruct-2507")

def to_text(ex):
    return {"text": tok.apply_chat_template(ex["messages"], tokenize=False)}

train = ds["train"].map(to_text)

Fine-tuning notes

• —Some source articles are long — use a generous max_seq_length (8192–16384) and watch your trainer logs for truncation warnings (e.g. Unsloth's "Removed N samples ... no response found after truncation").
• —The dataset ships its own test split, so you can evaluate directly on it instead of carving an eval set from train.

Licensing & attribution

This is a derivative aggregation. Each row retains a source_dataset field for attribution. Licensing follows the respective upstream source datasets — review the source dataset cards before commercial use. The license: other tag reflects this mixed/per-source provenance.

Citation

If you use this dataset, please cite the upstream sources listed in Source composition above.