CoolFace
Datasetpublic

saidutta69/Odia-Web-Corpus-v3

Odia Web Corpus v3 Third iteration of the Odia web corpus with enhanced deduplication, quality filtering, and standardized Parquet splits for pretraining and evaluation. Dataset Details Language: Odia (ISO 639-3: or) Format: Parquet (columnar, compressed) Splits: train (900K), validation (50K), test (50K) License: CC-BY-SA-4.0 Data Fields Field Type Description text string Cleaned and filtered document text… See the full description on the dataset page: https://huggingface.co/datasets/saidutta69/Odia-Web-Corpus-v3.

sourceHugging Facecc-by-sa-4.0updated 11d agoView on Hugging Face
0likes64downloads
Dataset Card

Odia Web Corpus v3

<div align="center"> <img src="https://photu.kashyalabanavli.site/racer-is-op.png" alt="RACER IS OP" width="100%"> </div>

<br>

Third iteration of the Odia web corpus with enhanced deduplication, quality filtering, and standardized Parquet splits for pretraining and evaluation.

Dataset Details

  • —Language: Odia (ISO 639-3: or)
  • —Format: Parquet (columnar, compressed)
  • —Splits: train (900K), validation (50K), test (50K)
  • —License: CC-BY-SA-4.0

Data Fields

FieldTypeDescription
textstringCleaned and filtered document text

Usage

python
from datasets import load_dataset

ds = load_dataset("saidutta69/Odia-Web-Corpus-v3")
print(ds["train"][0]["text"][:200])

Citation

bibtex
@misc{odia-web-corpus-v3,
  author = {Sai Dutta},
  title = {Odia Web Corpus v3},
  year = {2024},
  publisher = {Hugging Face},
  url = {https://huggingface.co/datasets/saidutta69/Odia-Web-Corpus-v3}
}