Datasetpublic

KeisukeMiyamoto/lambda-corpus

lambda-corpus lambda-corpus is a Japanese text corpus for language model pretraining. It combines openly available Japanese datasets into a consistent format and provides predefined train, validation, and test splits. Purpose The dataset is intended for pretraining of Japanese language models. It contains web documents, Wikipedia-derived text, academic grant records, and synthetic question-answer text. Source Data Source Rows Tokens License… See the full description on the dataset page: https://huggingface.co/datasets/KeisukeMiyamoto/lambda-corpus.

sourceHugging Faceotherupdated 1mo agoView on Hugging Face

0likes276downloads

Dataset Card

lambda-corpus

lambda-corpus is a Japanese text corpus for language model pretraining. It combines openly available Japanese datasets into a consistent format and provides predefined train, validation, and test splits.

Purpose

The dataset is intended for pretraining of Japanese language models. It contains web documents, Wikipedia-derived text, academic grant records, and synthetic question-answer text.

Source Data

Source	Rows	Tokens	License
KeisukeMiyamoto/CleanedFineWeb2Edu-jp	11,772,204	7.61B	ODC-BY
hpprc/jawiki-bullet-points	3,523,358	1.55B	CC BY-SA 4.0
kanhatakeyama/CommonCrawl-RAG-QA-Calm3-22b-chat	2,525,793	1.04B	Other
hpprc/llmjp-kaken	1,079,030	1.03B	CC BY 4.0

Dataset Size

Split	Rows
train	18,522,556
validation	189,174
test	188,655
total	18,900,385

The dataset contains approximately 11.23B tokens measured with the gemma 4 tokenizer.

Columns

Column	Description
`id`	Identifier inherited from the source dataset
`text`	Normalized document text
`output_tokens`	Number of tokens in `text`, measured with the gemma 4 tokenizer
`source`	Name of the source dataset

Example

python

from datasets import load_dataset

dataset = load_dataset("KeisukeMiyamoto/lambda-corpus")
print(dataset["train"][0])

Support Lambda

Lambda is an open-source project for building small Japanese language models from scratch. As a student, I have funded this project with income from my part-time job, but the growing training costs are becoming difficult to cover.

Your support helps cover GPU costs and develop larger models. Thank you for helping Lambda continue to grow.

Vast.ai

Vast.ai offers affordable cloud GPUs for AI training, with NVIDIA H100 SXM GPUs available from around $1.54 per hour. If you purchase credits through the link below, I receive 3% in GPU credits at no extra cost to you.

https://cloud.vast.ai/?ref_id=521936

Ko-fi

Support Lambda with a donation starting from $5.