rustemgareev/mdeberta-v3-base-lite

mdeberta-v3-base-lite

This model was created through vocabulary pruning of the original microsoft/mdeberta-v3-base model while maintaining full quality for Latin and Cyrillic-based languages.

Supported Languages

• —Bulgarian
• —English
• —French
• —German
• —Russian
• —Spanish
• —Swahili
• —Turkish
• —Vietnamese

Usage

from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("rustemgareev/mdeberta-v3-base-lite")
model = AutoModel.from_pretrained("rustemgareev/mdeberta-v3-base-lite")

# Example usage
text = "This is an example text in English."
inputs = tokenizer(text, return_tensors="pt")
outputs = model(**inputs)

# Verify that the model outputs hidden states (embeddings)
print("Success! Output shape (batch_size, sequence_length, hidden_size):")
print(outputs.last_hidden_state.shape)

Performance Evaluation

Size Comparison

VRAM Usage Comparison

Estimated using [Hugging Face Accelerate Model Estimator](https://huggingface.co/docs/accelerate/main/en/usage_guides/model_size_estimator).

Semantic Similarity Comparison

Evaluation Method: Cosine similarity between embeddings of parallel sentences in different languages, using English as reference.

Test Phrases Used:

• —English: "Artificial intelligence learns to understand human languages and helps people communicate."
• —Bulgarian: "Изкуственият интелект се учи да разбира човешките езици и помага на хората да общуват."
• —French: "L'intelligence artificielle apprend à comprendre les langages humains et aide les gens à communiquer."
• —German: "Künstliche Intelligenz lernt, menschliche Sprachen zu verstehen und hilft Menschen bei der Kommunikation."
• —Russian: "Искусственный интеллект учится понимать человеческие языки и помогает людям общаться."
• —Spanish: "La inteligencia artificial aprende a entender los idiomas humanos y ayuda a las personas a comunicarse."
• —Swahili: "Akili ya kisasa inajifunza kuelewa lugha za wanadamu na kusaidia watu kuwasiliana."
• —Turkish: "Yapay zeka, insan dillerini anlamayı öğrenir ve insanların iletişim kurmasına yardımcı olur."
• —Vietnamese: "Trí tuệ nhân tạo học cách hiểu ngôn ngữ con người và giúp mọi người giao tiếp."

Similarity Results:

License

This model is distributed under the MIT License.