kravchenko/uk-mt5-base

The aim is to compress the mT5-base model to leave only the Ukrainian language and some basic English.

Reproduced the similar result (but with another language) from this medium article.

Results:

• —582M params -> 244M params (58%)
• —250K tokens -> 30K tokens
• —2.2GB size model -> 0.95GB size model