Datasetpublic

tvu-vlinhd11/pretrain-dataset-T2048-13B

Pretrain Dataset (Tokenized) This dataset contains tokenized and packed sequences ready for LLM pretraining. Dataset Details Property Value Sequences 6,474,097 Sequence Length 2048 Tokenizer ./vn_spm_v3_fast2/ Total Tokens 13,258,950,332 Shards 13 Created 2025-12-10 Dataset Structure Each sample contains: input_ids: List of token IDs (length: 2048) attention_mask: Attention mask (1 for real tokens, 0 for padding)… See the full description on the dataset page: https://huggingface.co/datasets/tvu-vlinhd11/pretrain-dataset-T2048-13B.

sourceHugging Faceapache-2.0updated 10mo agoView on Hugging Face

0likes353downloads

Dataset Card

Pretrain Dataset (Tokenized)

This dataset contains tokenized and packed sequences ready for LLM pretraining.

Dataset Details

Property	Value
Sequences	6,474,097
Sequence Length	2048
Tokenizer	`./vn_spm_v3_fast2/`
Total Tokens	13,258,950,332
Shards	13
Created	2025-12-10

Dataset Structure

Each sample contains:

—input_ids: List of token IDs (length: 2048)
—attention_mask: Attention mask (1 for real tokens, 0 for padding)

Usage

python

from datasets import load_dataset

dataset = load_dataset("tvu-vlinhd11/pretrain-dataset-T2048-13B")
train_data = dataset["train"]
sample = train_data[0]
input_ids = sample["input_ids"]
attention_mask = sample["attention_mask"]

License

Apache 2.0