CoolFace
Modelpublic

clue/roberta_chinese_3L312_clue_tiny

sourceHugging Faceupdated 5y agoView on Hugging Face
2likes111downloads
Model Card

Introduction

This model was trained on TPU and the details are as follows:

Model

Model_nameparamssizeTraining_corpusVocab
`RoBERTa-tiny-clue` <br/>Supersmallmodel7.5M28.3MCLUECorpus2020CLUEVocab
`RoBERTa-tiny-pair` <br/>Supersmallsentencepairmodel7.5M28.3MCLUECorpus2020CLUEVocab
`RoBERTa-tiny3L768-clue` <br/>small_model38M110MCLUECorpus2020CLUEVocab
`RoBERTa-tiny3L312-clue` <br/>small_model<7.5M24MCLUECorpus2020CLUEVocab
`RoBERTa-large-clue` <br/> Large_model290M1.20GCLUECorpus2020CLUEVocab
`RoBERTa-large-pair` <br/>Largesentencepair_model290M1.20GCLUECorpus2020CLUEVocab

Usage

With the help ofHuggingface-Transformers 2.5.1, you could use these model as follows

tokenizer = BertTokenizer.from_pretrained("MODEL_NAME")
model = BertModel.from_pretrained("MODEL_NAME")

MODEL_NAME：

Model_NAMEMODEL_LINK
RoBERTa-tiny-clue`clue/roberta_chinese_clue_tiny`
RoBERTa-tiny-pair`clue/roberta_chinese_pair_tiny`
RoBERTa-tiny3L768-clue`clue/roberta_chinese_3L768_clue_tiny`
RoBERTa-tiny3L312-clue`clue/roberta_chinese_3L312_clue_tiny`
RoBERTa-large-clue`clue/roberta_chinese_clue_large`
RoBERTa-large-pair`clue/roberta_chinese_pair_large`

Details

Please read <a href='https://arxiv.org/pdf/2003.01355'>https://arxiv.org/pdf/2003.01355.

Please visit our repository: https://github.com/CLUEbenchmark/CLUEPretrainedModels.git