datasets
Training and evaluation data, with the modality, task and licence stated up front. Listed live from the Hugging Face Hub.
Assamese-Text-Dataset-45T-Tokens
I have massive Assamese Dataset nearly about 45.3T (45333004592600) Tokens
It has a lots of Assamese sentances from various sources, 99.9999% of the dataset are cleanned
just download the backup_data.tar.zst file and start using it.
happy training....
My email: ranjitdax89@gmail.com
At least share your opinion… or maybe a simple “thanks” 😄
Topic / Dataset
Tokens
Approx. Scale
Source
Poems Dataset
92.6K
0.0000926B… See the full description on the dataset page: https://huggingface.co/datasets/Ranjit89/Assamese-Text-Dataset-45T-Tokens.xahitya-assamese-corpus
Xahitya Assamese Corpus
A large-scale Assamese literary text corpus scraped from Xahitya.org, containing Assamese prose, essays, stories, poems, and other long-form literary writings.
This dataset is intended for:
Assamese NLP research
Language model pretraining
Tokenizer training
Text generation
Linguistic analysis
Low-resource language AI research
Dataset Structure
The dataset currently contains:
xahitya_dump/
├── articles.jsonl
└── corpus.txt
happy… See the full description on the dataset page: https://huggingface.co/datasets/Ranjit89/xahitya-assamese-corpus.assamese-monolingual-corpus
Assamese Monolingual Corpus (2025)
A high-quality, sentence-level Assamese monolingual dataset containing 1.61 million cleaned, segmented, and deduplicated sentences in Bengali script. This corpus supports Assamese NLP development, language modeling, and public deployment for Northeast India.
Dataset Summary
Language: Assamese (Bengali script)
Size: 1,613,879 sentences
Format: Plain text CSV (text column)
Total tokens: 77,427,585 (using IndicBERTv2 tokenizer)… See the full description on the dataset page: https://huggingface.co/datasets/MWirelabs/assamese-monolingual-corpus.assamese-monolingual-corpus
Assamese Monolingual Corpus (2025)
A high-quality, sentence-level Assamese monolingual dataset containing 1.61 million cleaned, segmented, and deduplicated sentences in Bengali script. This corpus supports Assamese NLP development, language modeling, and public deployment for Northeast India.
Dataset Summary
Language: Assamese (Bengali script)
Size: 1,613,879 sentences
Format: Plain text CSV (text column)
Total tokens: 77,427,585 (using IndicBERTv2 tokenizer)… See the full description on the dataset page: https://huggingface.co/datasets/jintz0/assamese-monolingual-corpus.assamese_wikipedia
Assamese Wikipedia Corpus
Dataset Description
The Assamese Wikipedia Corpus is a pure Assamese text dataset derived from the Assamese-language Wikipedia (as.wikipedia.org).
It contains cleaned plain text extracted from Wikipedia articles, stripped of all formatting, with non-Assamese characters completely removed.
This dataset is designed for language modeling, NLP research, creating Assamese specific tokenizers, and other Assamese-language processing tasks.… See the full description on the dataset page: https://huggingface.co/datasets/marsh-mellow/assamese_wikipedia.assamese-wiki-corpus
Assamese Wiki Corpus (ananddey/assamese-wiki-corpus)
A clean, large scale Assamese text corpus spanning wiki articles, literary works, dictionary entries, and quotations, curated for language model pre training, fine tuning, and NLP research.
Total characters: 65,899,040Approximate tokens : 16,474,760 (16.5M)
Fields
Field
Type
Description
id
int64
Wikimedia page ID
title
string
Page title
text
string
Cleaned plain text content
source
class_label… See the full description on the dataset page: https://huggingface.co/datasets/ananddey/assamese-wiki-corpus.assamese-sft-dataset-v1
Assamese SFT Dataset v1 (ananddey/assamese-sft-dataset-v1)
An industry-standard, curated, and deduplicated Supervised Fine-Tuning (SFT) dataset for training Assamese language models and conversational AI assistants.
📊 Dataset Summary
Total Samples: 61,929 instruction-response pairs
Train Split: 58,833 samples
Validation Split: 3,096 samples
Languages: Assamese (as), English (en)
Primary Use Case: SFT / Instruction Fine-Tuning for generative LLMs (e.g. Gemma… See the full description on the dataset page: https://huggingface.co/datasets/ananddey/assamese-sft-dataset-v1.assamese-news-corpus
Assamese News Corpus
A cleaned corpus of 1,369 Assamese language news articles for training of Assamese language models.
Splits
Split
Articles
train
1,298
validation
71
Fields
Field
Type
Description
title
string
Article headline in Assamese
text
string
Cleaned article body text
Usage
from datasets import load_dataset
dataset = load_dataset("ananddey/assamese-news-corpus")
print(dataset["train"][0])… See the full description on the dataset page: https://huggingface.co/datasets/ananddey/assamese-news-corpus.
