izzako/balinese-pixelgpt

Balinese PixelGPT Dataset This dataset contains preprocessed Balinese text data for training PixelGPT models. Dataset Statistics Language: Balinese (bali) Total samples: 54,467 Train samples: 54,017 Test samples: 450 Tokenizers Grapheme tokenizer: izzako/javanese-llama-tokenizer LLaMA tokenizer: ernie-research/DualGPT Features text_id: Document identifier chunk_id: Chunk identifier within document pixel_values: Rendered pixel… See the full description on the dataset page: https://huggingface.co/datasets/izzako/balinese-pixelgpt.