izzako/balinese-pixelgpt

Balinese PixelGPT Dataset

This dataset contains preprocessed Balinese text data for training PixelGPT models.

Dataset Statistics

• —Language: Balinese (bali)
• —Total samples: 54,467
• —Train samples: 54,017
• —Test samples: 450

Tokenizers

• —Grapheme tokenizer: izzako/javanese-llama-tokenizer
• —LLaMA tokenizer: ernie-research/DualGPT

Features

• —text_id: Document identifier
• —chunk_id: Chunk identifier within document
• —pixel_values: Rendered pixel representation of aksara text
• —grapheme_token_ids: Token IDs from grapheme-based tokenizer (with EOS token)
• —llama_token_ids: Token IDs from LLaMA tokenizer (with EOS token)
• —text: Original text chunk

Renderer Configuration

• —Renderer path: ../../renderers/m4_renderer

Usage

from datasets import load_dataset

# Load the dataset
dataset = load_dataset("izzako/balinese-pixelgpt")

# Access train and test splits
train_data = dataset['train']
test_data = dataset['test']

# Example: Get first sample
sample = train_data[0]
print(sample['text'])

Citation

If you use this dataset, please cite:

@dataset{balinese_pixelgpt,
  title = {Balinese PixelGPT Dataset},
  author = {Musa Izzanardi Wijanarko},
  year = {2025},
  publisher = {Hugging Face},
  url = {https://huggingface.co/datasets/izzako/balinese-pixelgpt}
}