lyeonii/bert-tiny

1---2license: mit3language:4- en5---6 7# BERT-Tiny (uncased)8This is the smallest version of 24 smaller BERT models (English only, uncased, trained with WordPiece masking)9released by [google-research/bert](https://github.com/google-research/bert).10 11These BERT models was released as TensorFlow checkpoints, however, this is the converted version to PyTorch.12More information can be found in [google-research/bert](https://github.com/google-research/bert) or [lyeoni/convert-tf-to-pytorch](https://github.com/lyeoni/convert-tf-to-pytorch).13 14## Evaluation15Here are the evaluation scores (F1/Accuracy) for the MPRC task.16|Model|MRPC|17|-|:-:|18|BERT-Tiny|81.22/68.38|19|BERT-Mini|81.43/69.36|20|BERT-Small|81.41/70.34|21|BERT-Medium|83.33/73.53|22|BERT-Base|85.62/78.19|23 24### References25```26@article{turc2019,27  title={Well-Read Students Learn Better: On the Importance of Pre-training Compact Models},28  author={Turc, Iulia and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina},29  journal={arXiv preprint arXiv:1908.08962v2 },30  year={2019}31}32```