AOYPSK/lao_pairs_final

🇱🇦 Lao SFT Pairs Final

A cleaned and merged Lao-language instruction-tuning dataset for supervised fine-tuning (SFT) of large language models — specifically built to improve Lao language capability in models like Gemma 4.

Dataset Summary

Data Sources

This dataset merges two sources:

1. Lao continuation corpus (32.5%) Real Lao text from Lao Wikipedia, Lao news sites (KPL, Vientiane Times), and other Lao web sources — normalized, cleaned, and formatted as text-continuation pairs. Teaches the model Lao fluency, vocabulary, and sentence structure.

2. Saillab Alpaca Lao (67.5%) Instruction-following pairs from `saillab/alpaca-lao-cleaned`, cleaned and filtered. Teaches the model to follow instructions and answer questions in Lao.

Task Types

Data Format

Each example is a JSON object with 5 fields:

{
  "instruction": "ນະຄອນຫຼວງຂອງ ສປປ ລາວ ແມ່ນຫຍັງ?",
  "input": "",
  "output": "ນະຄອນຫຼວງຂອງ ສາທາລະນະລັດ ປະຊາທິປະໄຕ ປະຊາຊົນລາວ ແມ່ນວຽງຈັນ.",
  "text": "### Instruction:\nນະຄອນຫຼວງຂອງ ສປປ ລາວ ແມ່ນຫຍັງ?\n\n### Input:\n\n### Response:\nນະຄອນຫຼວງຂອງ ສາທາລະນະລັດ ປະຊາທິປະໄຕ ປະຊາຊົນລາວ ແມ່ນວຽງຈັນ.",
  "source": "saillab_alpaca_lao"
}

The text field is the pre-formatted string for SFTTrainer — pass it directly via dataset_text_field="text".

Cleaning Applied

The following examples were removed before training:

• —English-only outputs — 554 removed (outputs with 0 Lao characters)
• —Copy examples — 160 removed (output identical to input)
• —`nan` inputs — cleaned (pandas artifact from source dataset)
• —Duplicates — 112 removed (same instruction + output)
• —Train/val leakage — 6 removed from val

Final Lao script ratio: 90% in instructions, 81% in outputs.

Usage

from datasets import load_dataset

ds = load_dataset("AOYPSK/lao_pairs_final")

# For SFTTrainer — use the pre-built text field
trainer = SFTTrainer(
    ...
    dataset_text_field = "text",
)

Or load locally:

from datasets import load_dataset

ds = load_dataset("json", data_files={
    "train":      "lao_train_final.jsonl",
    "validation": "lao_val_final.jsonl",
})

Intended Use

This dataset is designed for fine-tuning Gemma 4 (or similar models) to improve Lao language understanding and generation. It is best used as the SFT stage after continued pre-training (CPT) on a Lao corpus.

Recommended base model: `AOYPSK/gemma-4-lao-cpt-v1`

Training Example

from unsloth import FastLanguageModel
from datasets import load_dataset
from trl import SFTTrainer, SFTConfig
import torch

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name     = "AOYPSK/gemma-4-lao-cpt-v1",
    max_seq_length = 4096,
    load_in_4bit   = False,
    dtype          = torch.bfloat16,
)

model = FastLanguageModel.get_peft_model(
    model, r=64, lora_alpha=64,
    target_modules=["q_proj","k_proj","v_proj","o_proj",
                    "gate_proj","up_proj","down_proj"],
)

ds = load_dataset("AOYPSK/lao_pairs_final")

trainer = SFTTrainer(
    model         = model,
    tokenizer     = tokenizer,
    train_dataset = ds["train"],
    eval_dataset  = ds["validation"],
    args = SFTConfig(
        num_train_epochs   = 3,
        learning_rate      = 1e-4,
        bf16               = True,
        dataset_text_field = "text",
        max_seq_length     = 4096,
    ),
)
trainer.train()

Limitations

• —Instructions are a mix of pure Lao (~77%) and Lao-English mixed (~23%)
• —Content topics are general/global — limited Lao-specific cultural knowledge
• —Not suitable for tasks requiring deep Lao legal, medical, or government knowledge without additional domain-specific data

Contact

Dataset prepared by @AOYPSK. Portfolio : @AOYPSK PORTFOLIO.