mjbommar/magic-bert-dataset

Magic-BERT Binary File Classification Dataset This dataset contains tokenized binary file samples for training MIME type classification models. Each sample is a 64KB chunk from the beginning of a file, tokenized using a byte-level BPE tokenizer. Splits Split Samples Train 37,111 Validation 4,591 Test 4,748 Features Each sample contains: Feature Type Description blake2b string Content hash (unique sample ID) mime_type… See the full description on the dataset page: https://huggingface.co/datasets/mjbommar/magic-bert-dataset.