nahid-hub/B-CORE-bengali-corpus

B-CORE: Bangla Pretraining Corpus

B-CORE (Bengali Context-aware Optimized and Refined Entities) is a large-scale, rigorously curated Bangla monolingual corpus for language model pretraining, comprising 16.5 million documents (4.32 billion tokens, 52GB (20.8 GB Compressed)). It is among the largest and most carefully curated Bangla pretraining corpora available, constructed through a reproducible multi-stage pipeline.

B-CORE was used to pretrain the **BnLM-F** and **BnLM-C** Bengali efficient pretrained models suite from scratch. See the full BLUGE collection for the complete release — evaluation benchmark, tokenizers, and all three pretrained models.

Dataset Description

B-CORE is built via a systematic quality-filtering and cross-corpus deduplication pipeline, achieving a 22.4% reduction in corpus volume relative to raw source data. This filtering removes low-quality, duplicate, and noisy text before pretraining, rather than relying on raw scraped volume — the corpus is designed around the premise that principled curation matters more than scale alone for low-resource language pretraining.

Dataset Structure

B-CORE is distributed as unlabeled, plain-text documents — there is a single train split, since this corpus is intended for pretraining rather than task evaluation. For evaluation data, see the BLUGE benchmark.

Fields:

• —text — the cleaned, deduplicated Bangla document text
• —id — unique document identifier

Usage

from datasets import load_dataset

ds = load_dataset("nahid-hub/B-CORE-bengali-corpus", split="train")
print(ds[0])

Stream the corpus without downloading it fully (recommended given its size):

from datasets import load_dataset

ds = load_dataset("nahid-hub/B-CORE-bengali-corpus", split="train", streaming=True)
for example in ds:
    print(example["text"])
    break

Or read the Parquet shards directly with pandas:

import pandas as pd

df = pd.read_parquet("hf://datasets/nahid-hub/B-CORE-bengali-corpus/data/train-0000.parquet")

License

Released under CC BY 4.0. You are free to share and adapt this dataset for any purpose, including commercially, provided you give appropriate credit. Note that individual source components may carry their own attribution requirements — see Source Data above.

Citation

If you use this dataset, please cite:

@ARTICLE{BnLM-BLUGE-B-CORE,
  author={Hossain, Nahid and Faisal Kabir, Md.},
  journal={IEEE Access}, 
  title={Efficient Monolingual Pretraining in Low-Resource Settings Through Morphology-Aware Tokenization, Principled Corpus Denoising, and Benchmark-Driven Evaluation}, 
  year={2026},
  volume={14},
  number={},
  pages={91979-92003},
  keywords={Modeling;Multilingual;Training;Cleaning;Vocabulary;Labeling;Tokenization;Computational linguistics;Pipelines;Conferences;B-CORE;BLUGE;BnLM;corpus;evaluation benchmark;pretrained models},
  doi={10.1109/ACCESS.2026.3701520}
}