sabin1234/NEPSE_Dividend_FAQ_Dataset_Romanized_Nepali_Questions

NEPSE Dividend FAQ Dataset (Romanized Nepali Questions)

File: nepse_dividend_faq_romanized.jsonl Total records: 2,000 Format: JSON Lines (.jsonl) — one JSON object per line Language: Nepali (ne / ISO 639-3 npi), answers in Devanagari script (Deva); questions in romanized Nepali (Latin letters) Domain: Financial services — NEPSE (Nepal Stock Exchange) historical dividend records Task type: Instruction-following (instruction-following) Generation type: Real (real) — this is factual data derived from actual company dividend history, not synthetically generated text License: Apache-2.0 (permissive)

This dataset is a single-turn (1 question + 1 answer) factual FAQ dataset built from real historical dividend announcements of companies listed on the Nepal Stock Exchange (NEPSE). Every question asks for the total dividend percentage (or bonus share percentage) that a specific listed company declared for a specific Nepali fiscal year (Bikram Sambat, e.g. 2077/2078), and every answer states the factual dividend percentage. A distinctive feature of this dataset is that questions are written in romanized Nepali (Latin script) while answers are written in standard Nepali (Devanagari script) — making it useful for romanized-input / Devanagari-output transliteration-aware QA systems.

1. Table of Contents

2. Dataset Summary

3. File / Record Schema

Each line of the .jsonl file is a single, independent JSON object with the following 19 top-level fields:

3.1 Nested schema: the conversations array

Each element of conversations is an object with exactly two fields:

4. Nested metadata_json Schema

The metadata_json field is a JSON-encoded string (must be parsed with a second json.loads() call) containing 9 keys. These values are constant across all 2,000 records in this file — i.e., the entire dataset represents one single, narrowly-scoped generation configuration.

Note: Although english_content_allowed is false and content_script declares Devanagari, the question side of every conversation is actually written in romanized Latin-script Nepali — the false/देवनागरी metadata describes the intended answer content policy, not the literal question text. This is an important nuance to document (see Section 16).

5. Conversation Structure

Unlike multi-turn conversational datasets, this dataset is strictly single-turn: every record contains exactly 2 messages.

6. Question Pattern Diversity

Every question is built from one of 3 phrasing templates, applied to one of 2 dividend types, giving 6 total surface-form templates, each filled in with a specific company code and fiscal year. All 2,000 questions fall cleanly into these 6 buckets.

6.1 The 3 phrasing templates (using <CO> = company code, <FY> = fiscal year)

For the bonus-share dividend type, the same 3 templates are used with kul labhansha ("total dividend") replaced by bonas seyar (labhansha) ("bonus share (dividend)"):

6.2 Distribution across the 6 question forms

7. Answer Pattern Diversity

Unlike the free-form assistant answers in narrative health-style datasets, every answer here follows one single fixed sentence template, varying only in the company code, fiscal year, and numeric percentage:

8. Company Coverage

The dataset references 210 unique NEPSE-listed companies, each identified by a romanized letter-by-letter spelling of its stock ticker symbol (e.g. the ticker "SLBSL" is spelled out phonetically as esielabiesaela, "S-L-B-S-L"). Company names are not given in expanded/full form anywhere in the dataset — only these phonetic ticker spellings are used.

8.1 Most frequently referenced companies (top 15 by record count)

The remaining ~195 companies appear with lower frequency, down to companies referenced only once. Because tickers are spelled phonetically letter-by-letter rather than given as standard stock symbols, mapping a given code back to a real-world company name requires manually decoding each letter (e.g., e = "E", bi = "B", si = "C", di = "D", ela = "L", etc.) — see Section 16 for a caveat on this.

9. Fiscal Year Coverage

Fiscal years are expressed in the Bikram Sambat (BS) calendar, in YYYY/YYYY format (e.g. 2077/2078 corresponds roughly to 2020/2021 in the Gregorian calendar). The dataset spans 18 unique fiscal years.

10. Dividend Value Coverage

Dividend percentages are stated as plain numbers followed by %, and include both whole numbers and precise decimal values (a result of real-world dividend calculations, e.g. splitting a cash dividend across share counts).

10.1 Top 15 most frequent dividend values

The presence of 0% as the third most common value (114 records) is notable — it correctly represents years in which a company declared no dividend, confirming this is real, unfiltered historical data rather than a cherry-picked "success stories only" dataset.

11. Length & Size Statistics

All lengths measured in characters.

12. Metadata Field Value Reference

Quick reference for every metadata field that is constant across the dataset (useful for filtering scripts, even though the value never varies here):

Fields that vary per record:

13. Loading the Dataset (Code Demos)

13.1 Plain Python — streaming line-by-line (no dependencies)

import json

path = "nepse_dividend_faq_romanized.jsonl"

records = []
with open(path, encoding="utf-8") as f:
    for line in f:
        line = line.strip()
        if not line:
            continue
        record = json.loads(line)
        records.append(record)

print(f"Loaded {len(records)} records")

# Print the first Q&A pair, plus its parsed nested metadata
first = records[0]
question = first["conversations"][0]["value"]
answer = first["conversations"][1]["value"]
print("Q:", question)
print("A:", answer)

metadata = json.loads(first["metadata_json"])  # metadata_json is a JSON-encoded string
print("Behavior:", metadata["behavior"])

13.2 Using pandas

import pandas as pd
import json

path = "nepse_dividend_faq_romanized.jsonl"

df = pd.read_json(path, lines=True)
print(df.shape)              # (2000, 19)
print(df.columns.tolist())   # all 19 top-level fields

# Split question/answer into their own columns
df["question"] = df["conversations"].apply(lambda c: c[0]["value"])
df["answer"] = df["conversations"].apply(lambda c: c[1]["value"])

# Parse the nested metadata_json string into real columns
meta_df = df["metadata_json"].apply(json.loads).apply(pd.Series)
df = pd.concat([df.drop(columns=["conversations", "metadata_json"]), df[["question", "answer"]], meta_df], axis=1)

df.to_csv("nepse_dividend_flat.csv", index=False, encoding="utf-8-sig")
print(df.head())

13.3 Extracting structured (company, fiscalyear, dividendtype, percentage) tuples

import json
import re

fy_pattern = re.compile(r"20\d{2}/20\d{2}")
pct_pattern = re.compile(r"\d+(?:\.\d+)?%")

def parse_answer(answer_text: str):
    """Extract company code, fiscal year, dividend type, and percentage from a gpt answer."""
    fiscal_year = fy_pattern.search(answer_text).group()
    percentage = pct_pattern.search(answer_text).group()
    dividend_type = "bonus_share" if "bonas seyar" in answer_text else "total"
    company_code = answer_text.split(" ko aarthik barsha")[0]
    return {
        "company_code": company_code,
        "fiscal_year": fiscal_year,
        "dividend_type": dividend_type,
        "percentage": percentage,
    }

with open("nepse_dividend_faq_romanized.jsonl", encoding="utf-8") as f:
    records = [json.loads(line) for line in f]

structured = [parse_answer(r["conversations"][1]["value"]) for r in records]
print(structured[:5])

13.4 Using Hugging Face datasets

from datasets import load_dataset

dataset = load_dataset(
    "json",
    data_files="nepse_dividend_faq_romanized.jsonl",
    split="train",
)

print(dataset)
print(dataset[0]["conversations"])

# Filter to records about bonus share dividends only
bonus_share = dataset.filter(
    lambda r: "bonas seyar" in r["conversations"][0]["value"]
)
print(len(bonus_share))  # 683

# Filter to a single fiscal year
fy_2081_82 = dataset.filter(
    lambda r: "2081/2082" in r["conversations"][0]["value"]
)
print(len(fy_2081_82))  # 206

13.5 Converting to a chat-fine-tuning message format

import json

def to_chat_messages(record):
    role_map = {"human": "user", "gpt": "assistant"}
    return [
        {"role": role_map[turn["from"]], "content": turn["value"]}
        for turn in record["conversations"]
    ]

with open("nepse_dividend_faq_romanized.jsonl", encoding="utf-8") as f:
    records = [json.loads(line) for line in f]

chat_formatted = [to_chat_messages(r) for r in records]

with open("nepse_dividend_chat_format.jsonl", "w", encoding="utf-8") as out:
    for messages in chat_formatted:
        out.write(json.dumps({"messages": messages}, ensure_ascii=False) + "\n")

14. Example Records

14.1 Total dividend example

{
  "id": "nepse_dividend_0001",
  "conversations": [
    {"from": "human", "value": "aarthik barsha 2077/2078 ma esielabiesaela ko kul labhansha kati thiyo?"},
    {"from": "gpt", "value": "esielabiesaela ko aarthik barsha 2077/2078 ko kul labhansha 21% thiyo."}
  ],
  "source": "NEPSE Dividend History",
  "source_name": "nepse_dividend_faq",
  "source_repo": "NEPSE Dividend History",
  "source_config": "default",
  "source_split": "train",
  "source_revision": "v1",
  "source_row_id": "nepse_dividend_0001",
  "language": "ne",
  "language_code": "npi",
  "script": "Deva",
  "license": "Apache-2.0",
  "license_tier": "permissive",
  "task_type": "instruction-following",
  "generation_type": "real",
  "condition": "real",
  "url": "",
  "metadata_json": "{\"generation_domain\": \"वित्तीय सेवा\", \"generation_category\": \"नेप्से लाभांश इतिहास\", \"generation_sub_domain\": \"कम्पनी लाभांश\", \"behavior\": \"तथ्यात्मक लाभांश जानकारी दिने\", \"behavior_definition\": \"निर्दिष्ट कम्पनी र आर्थिक वर्षको लाभांश तथ्यांक प्रयोग गरी प्रश्नको प्रत्यक्ष उत्तर दिने।\", \"question_type\": \"तथ्यमा आधारित प्रश्न\", \"content_language\": \"नेपाली\", \"content_script\": \"देवनागरी\", \"english_content_allowed\": false}"
}

14.2 Bonus share dividend example

{
  "id": "nepse_dividend_0027",
  "conversations": [
    {"from": "human", "value": "esielabiesaela ko aarthik barsha 2077/2078 ko bonas seyar labhansha pratishat kati ho?"},
    {"from": "gpt", "value": "esielabiesaela ko aarthik barsha 2077/2078 ko bonas seyar labhansha 21% thiyo."}
  ]
}

(Non-varying metadata fields omitted for brevity — see Section 12 for the full fixed values.)

15. Suggested Use Cases

16. Limitations & Notes

17. Quick Reference Cheat Sheet