somasekhar-dev/nexttoken-pmkisan-domain-sft-data

NextToken pmkisan domain SFT data (v1)

Grounded multilingual QA dataset for fine-tuning `somasekhar-dev/NextToken-model-1` on the Indian government-schemes / banking-financial domain.

Generated by a pipeline (chunk source docs -> generate questions -> generate grounded answers -> validate/assemble) using a local LLM generator, from ~57 scheme/product source documents (PM-KISAN, Ayushman Bharat, MGNREGA, banking products, insurance, savings instruments, etc.).

Files

• —train.jsonl (4,728 rows), val.jsonl (167 rows), test.jsonl (358 rows) -- split by source document, not by example, so val/test measure generalization to unseen documents rather than memorized chunks.
• —report.json -- per-language, per-scheme, and split counts.

Row schema

{
  "id": "...",
  "scheme": "PM-KISAN",
  "question_type": "eligibility_check",
  "query_lang": "kn",
  "doc_lang": "en",
  "answer_lang": "kn",
  "context": "...",
  "question": "...",
  "answer": "...",
  "answerable": true,
  "verified": null,
  "source_url": "...",
  "chunk_id": "...",
  "context_tokens": 210,
  "question_tokens": 14,
  "answer_tokens": 72,
  "total_tokens": 326
}

answerable: false rows (~4% of the set) are grounded-refusal examples -- the question isn't answerable from the given context, and the target teaches the model to say so rather than fabricate.

Known limitations (v1)

• —Machine-generated by a single generator model, template-driven question types (direct_fact, eligibility_check, process, negation_edge_case) -- not independently fact-checked at scale (the generation pipeline has a --verify LLM fact-check pass, not run for this v1).
• —Source pool is thin relative to the pipeline's own target: ~187 chunks (after reserving a held-out document set for a separate eval benchmark) feeding 4,728 training examples -- meaningful phrasing/template reuse across languages per chunk. A model fine-tuned 3 epochs on this v1 set showed clear overfitting (val loss rising while train loss kept falling) -- more source-document diversity is the most likely fix, ahead of further training-hyperparameter tuning.
• —9 languages are roughly balanced (~580-590 examples each) by construction, not by natural distribution.

Language coverage

hi, kn, ta, te, mr, bn, gu, pa, en (~580-590 examples each)