ChamaraVishwajithRajapaksha/sinhala-text-dataset

Sinhala Continuous Pretraining Corpus

Curated by [HelaAI](https://huggingface.co/HelaAI)

Dataset Summary

This dataset is a Sinhala-language text corpus assembled for continuous pretraining of language models. It combines multiple sources into a single, cleaned, block-structured corpus:

1. 1.News articles — Sinhala news text extracted from the article_sinhala field of Hamza-Ziyard/CNN-Daily-Mail-Sinhala.
2. 2.O/L Sinhala Buddhism — Sinhala-medium educational text covering the GCE Ordinary Level (O/L) Buddhism syllabus.
3. 3.O/L Mathematics — Sinhala-medium educational text covering the GCE Ordinary Level (O/L) Mathematics syllabus.

The goal of the dataset is to give a language model broader exposure to Sinhala across journalistic, religious/cultural, and academic/STEM registers, rather than news text alone, which improves vocabulary and domain coverage for downstream Sinhala NLP tasks.

Note: Please update the exact source details for the O/L Buddhism and Mathematics portions (origin, whether scanned/OCR'd or digitally sourced, and licensing) before publishing — these are marked as placeholders below since the specifics weren't provided.

Supported Tasks

• —Continuous / domain-adaptive pretraining of causal or masked language models on Sinhala text.
• —Auxiliary use for Sinhala tokenizer training, language modeling benchmarks, and text corpus research.

This dataset is not structured for supervised tasks like summarization or QA (no input/target pairs) — it is raw continuous text.

Languages

• —Sinhala (`si`) — the entire corpus is monolingual Sinhala.

Dataset Structure

Data Instances

Each row/record is a training block made up of multiple concatenated paragraphs, separated by double newlines, e.g.:

{"text": "පළමු ඡේදය...\n\nදෙවන ඡේදය...\n\n...\n\nවිසිවන ඡේදය..."}

Data Fields

Data Splits

Total download size (Parquet, compressed): ~204 MB.

If you need eval splits, we recommend holding out a random 1–2% of blocks before training.

Dataset Creation

Source Data

Processing Pipeline

The news portion was built with the following steps (see create_dataset.py):

1. 1.Load the source dataset via datasets.load_dataset.
2. 2.Extract the article_sinhala column from each row.
3. 3.Clean each article: strip leading/trailing whitespace and collapse internal whitespace/newlines into single spaces.
4. 4.Filter out any article shorter than MIN_TEXT_LENGTH (50 characters).
5. 5.Group cleaned paragraphs into blocks of PARAGRAPHS_PER_BLOCK (20) paragraphs each, joined with \n\n.
6. 6.Export the blocks as:
7. 7.continuous_pretrain_sinhala.txt — plain text, blocks separated by \n\n====================\n\n
8. 8.continuous_pretrain_sinhala.jsonl — one JSON object per line, {"text": "..."}

The O/L Buddhism and Mathematics portions were processed the same way (paragraph extraction → whitespace cleanup → length filtering → 20-paragraph blocking) before being merged into the same text-only schema and appended to the JSONL/TXT outputs.

Annotations

No annotations are included — this is unlabeled raw text for language modeling only.

Considerations for Using the Data

Social Impact and Biases

• —News content reflects the editorial perspective and topic selection of its original publisher(s) and may carry political, cultural, or regional bias typical of Sri Lankan news media.
• —Religious (Buddhism) content reflects a specific curriculum's framing of Buddhist teachings and may not represent all Buddhist traditions or interpretations.
• —The corpus is educational-syllabus-aligned in parts (O/L level), so vocabulary/style may skew toward formal, textbook Sinhala rather than colloquial/spoken registers.

Other Known Limitations

• —No deduplication step is described in the pipeline — near-duplicate paragraphs (e.g., syndicated news) may be present.
• —No PII scrubbing was performed; news text may reference real named individuals, and O/L past-paper text may include names.
• —Block boundaries are arbitrary (every 20 paragraphs) and do not necessarily align with topic or document boundaries.

Licensing Information

(To be completed.) The news component derives from Hamza-Ziyard/CNN-Daily-Mail-Sinhala — check that dataset's license before redistribution. The O/L Buddhism and Mathematics content's license depends on its original source (e.g., Sri Lankan Department of Education materials, which may carry their own usage restrictions) and should be confirmed before this dataset is made public or used commercially.

Citation Information

If you use this dataset, please cite the upstream source(s):

@misc{cnn_dailymail_sinhala,
  title  = {CNN-Daily-Mail-Sinhala},
  author = {Hamza Ziyard},
  howpublished = {\url{https://huggingface.co/datasets/Hamza-Ziyard/CNN-Daily-Mail-Sinhala}}
}

(Add a citation entry for the O/L Buddhism and Mathematics source once identified.)

Dataset Card Contact

Maintained by HelaAI