Datasetpublic

Exqrch/javanese-Komodo-pixelgpt

Javanese PixelGPT Dataset This dataset contains preprocessed Javanese text data for training PixelGPT models. Dataset Statistics Language: Javanese (jawa) Total samples: 401,542 Train samples: 400,726 Test samples: 816 Tokenizers [TO BE EDITED] Features text_id: Document identifier chunk_id: Chunk identifier within document pixel_values: Rendered pixel representation of aksara text <tokenizer_name>_token_ids: Token IDs from… See the full description on the dataset page: https://huggingface.co/datasets/Exqrch/javanese-Komodo-pixelgpt.

sourceHugging Facecc-by-4.0updated 7mo agoView on Hugging Face

0likes98downloads

Dataset Card

Javanese PixelGPT Dataset

This dataset contains preprocessed Javanese text data for training PixelGPT models.

Dataset Statistics

—Language: Javanese (jawa)
—Total samples: 401,542
—Train samples: 400,726
—Test samples: 816

Tokenizers

—[TO BE EDITED]

Features

—text_id: Document identifier
—chunk_id: Chunk identifier within document
—pixel_values: Rendered pixel representation of aksara text
—<tokenizer_name>_token_ids: Token IDs from grapheme-based tokenizer (with EOS token)
—text: Original text chunk

Renderer Configuration

—Renderer path: ../../renderers/m4_renderer

Usage

python

from datasets import load_dataset

# Load the dataset
dataset = load_dataset("izzako/javanese-pixelgpt")

# Access train and test splits
train_data = dataset['train']
test_data = dataset['test']

# Example: Get first sample
sample = train_data[0]
print(sample['text'])

Citation

If you use this dataset, please cite:

bibtex

@dataset{javanese_pixelgpt,
  title = {Javanese PixelGPT Dataset},
  author = {Musa Izzanardi Wijanarko},
  year = {2025},
  publisher = {Hugging Face},
  url = {https://huggingface.co/datasets/izzako/javanese-pixelgpt}
}