saidutta69/odia_pretrain_dataset

Odia Pre-training Dataset

<div align="center"> <img src="https://photu.kashyalabanavli.site/racer-is-op.png" alt="RACER IS OP" width="100%"> </div>

<br>

10.32 million rows of Odia text. Zero access requests. Zero waiting. Zero gating nonsense.

The Origin Story (a.k.a. How a Pending Access Request Created a Monster)

It all started with a simple request: "Can I please access your Odia pretraining dataset?"

That was months ago. The access request is still pending.

So we did what any reasonable person would do when faced with institutional gatekeeping of a low-resource language's most important resource: we rebuilt the entire thing from scratch.

What began as a humble attempt to recreate a ~6M row dataset has now ballooned into a 10.32M rows (merged Odia-Web-Corpus-v5)er tabs, downloads every publicly available Odia corpus on the internet, writes deduplication scripts at 3 AM, spins up cloud VMs that keep getting preempted, fights with sandboxes that timeout, battles upload size limits, and emerges with a dataset that is 63% larger** than the one they were trying to access in the first place.

The original team wanted to gate access to protect their work. Instead, they inadvertently motivated the creation of a bigger, more diverse, and more openly accessible dataset. You're welcome.

Moral of the story: Gating your dataset doesn't protect it. It just makes someone build a better one out of spite.

What's In This Beast

Source Breakdown (all 25+ sources, because why stop at one)

What We Couldn't Get (Gated Datasets That Refused to Open)

If these ever get accepted, we'll merge them in too. Because apparently we can't stop.

Quality

We didn't just dump everything into a bucket. We actually cleaned it.

Data Schema

Quick Start

from datasets import load_dataset

# Just load it. No access form. No waiting. No emails.
dataset = load_dataset("saidutta69/odia_pretrain_dataset", split="train")

# Filter by source
news = dataset.filter(lambda x: x["source"] in ("varta", "odia_web_corpus_v5"))
instructions = dataset.filter(lambda x: x["type"] == "instruction")
parallel = dataset.filter(lambda x: x["source"] == "samanantar")

# Count by source
from collections import Counter
sources = Counter(x["source"] for x in dataset)
for src, count in sources.most_common():
    print(f"{src}: {count:,}")

License

CC-BY-4.0 - Use it commercially. Use it for research. Use it to train a model that competes with OdiaGenAI. We don't gate things here.

Acknowledgments

Special thanks to:

• —OdiaGenAI for accidentally motivating this entire project through the power of a pending access request
• —AI4Bharat for Samanantar and IndicNLP (openly accessible, unlike some)
• —BigScience for xp3 and the multilingual parallel data
• —DailyHunt/Varta for the news corpus
• —Every Odia NLP researcher who contributed open datasets
• —Coffee for keeping us awake during the 3 AM deduplication runs
• —Cloud VMs for getting killed mid-push exactly when it hurt the most

Citation

If this dataset helped your research, cite it:

@misc{odia_pretrain_dataset,
  author = {Saidutta Abhishek Dash},
  title = {Odia Pre-training Dataset: What Happens When You Gate a Low-Resource Language},
  year = {2025},
  publisher = {Hugging Face},
  note = {10.32M rows (merged Odia-Web-Corpus-v5) of Odia text, rebuilt from scratch because access requests went unanswered},
  howpublished = {\url{https://huggingface.co/datasets/saidutta69/odia_pretrain_dataset}},
}

"They gated the data. We un-gated the language."