1 results

encoded-corpus

inall models datasets apps agents people projects

Datasets

All datasets matching “encoded-corpus”

alibayram /cosmos-corpus-encoded Cosmos Corpus Encoded for Embedding Distillation This dataset is a pre-tokenized version of alibayram/cosmos-corpus-0-05-with-embeddings, designed for efficient embedding distillation training of MFT and TabiBERT models. Pre-processing & Filtering The dataset was processed using two different tokenizers to support multiple student architectures: MFT Tokenizer: A custom morphologically informed tokenizer. TabiBERT Tokenizer: A BERT-based tokenizer with 32k vocabulary.… See the full description on the dataset page: https://huggingface.co/datasets/alibayram/cosmos-corpus-encoded.textfeature-extraction100K<n<1M0 likes88 downloads8mo agoHugging Face