saidutta69/odia_pretrain_dataset_v2

Odia Pretrain Dataset v2

<div align="center"> <img src="https://photu.kashyalabanavli.site/racer-is-op.png" alt="RACER IS OP" width="100%"> </div>

<br>

12.3 million clean Odia documents. 30+ sources. Zero noise. Zero gating. Just data that actually works for pretraining.

The Origin Story (a.k.a. The Time We Filtered 98% of a Clean Dataset and Learned Our Lesson)

v1 was built from spite. v2 was built from more data.

We took v1 (25+ sources, 10.3M rows) as the foundation and asked: what else can we add?

• —monsoon-nlp/odia-cleaned -- fully deduplicated against v1 (0 new rows)
• —Wikipedia Odia -- 641 unique articles not already in v1
• —IndicCorpV2 Odia -- the heavyweight addition: 2 million unique rows of Odia text from AI4Bharat's collection

After deduplicating across all sources, we landed at 12.3M unique rows -- a 19% increase over v1 with minimal quality compromise.

The result is v2: v1's diversity plus everything new we could find, with nothing duplicated.

What's In This Beast

Source Breakdown

Data Schema

Quick Start

from datasets import load_dataset

# Load. No forms. No gatekeeping. Just data.
ds = load_dataset("saidutta69/odia_pretrain_dataset_v2", split="train")
print(f"Rows: {len(ds)}")

# Filter by source
monsoon = ds.filter(lambda x: x["source"] == "monsoon-nlp")
wikipedia = ds.filter(lambda x: x["source"] == "wikipedia")
indic = ds.filter(lambda x: x["source"] == "indiccorp_v2")

# Character length distribution
import statistics
lengths = [x["char_count"] for x in ds]
print(f"Mean: {statistics.mean(lengths):.0f}, Median: {statistics.median(lengths):.0f}")

License

CC-BY-4.0 -- Use it commercially. Use it for research. Train your Odia model. We don't gate things here.

Acknowledgments

• —monsoon-nlp for the original cleaned Odia corpus that did the hard work before we touched it
• —The person who called us out on the 98% filtration -- this dataset wouldn't exist without that reality check
• —Odia Wikipedia contributors for keeping the language alive online
• —AI4Bharat for IndicCorpV2
• —GCP VMs that got absolutely wrecked by 10M-row processing loops and kept coming back for more

Citation

@misc{odia_pretrain_dataset_v2,
  author = {Sai Dutta Abhishek Dash},
  title = {Odia Pretrain Dataset v2: The One That Keeps Its Promises},
  year = {2026},
  publisher = {Hugging Face},
  note = {10.3M rows of clean Odia text, minimal filtering, no gatekeeping},
  howpublished = {\url{https://huggingface.co/datasets/saidutta69/odia_pretrain_dataset_v2}},
}

"We filtered 98%. You told us we were wrong. We fixed it. This one's for you."