alibayram/cosmos-corpus-encoded
Cosmos Corpus Encoded for Embedding Distillation This dataset is a pre-tokenized version of alibayram/cosmos-corpus-0-05-with-embeddings, designed for efficient embedding distillation training of MFT and TabiBERT models. Pre-processing & Filtering The dataset was processed using two different tokenizers to support multiple student architectures: MFT Tokenizer: A custom morphologically informed tokenizer. TabiBERT Tokenizer: A BERT-based tokenizer with 32k… See the full description on the dataset page: https://huggingface.co/datasets/alibayram/cosmos-corpus-encoded.
Conversations for this repository live on Hugging Face.
CoolFace shows imported repositories read-only. Posting into someone else’s repository from here would need an authorised integration and the account holder’s consent, so the link goes to the source instead.
Open discussions on Hugging Face