NeuronUz/NeuronAI-Uzbek

1---2language:3  - uz4  - en5license: apache-2.06base_model: Qwen/Qwen3-4B7tags:8  - uzbek9  - qwen310  - language-model11  - text-generation12  - nlp13  - central-asia14  - low-resource15  - tokenizer-optimization16datasets:17  - behbudiy/alpaca-cleaned-uz18  - NeuronUz/uzbek-spelling-mcq19pipeline_tag: text-generation20model-index:21  - name: NeuronAI-Uzbek22    results:23      - task:24          type: text-generation25          name: Uzbek Language Understanding26        dataset:27          name: UzLiB Benchmark28          type: uzlib29        metrics:30          - type: accuracy31            value: 0.66232            name: Overall Accuracy33---34 35<div align="center">36 37# 🇺🇿 NeuronAI-Uzbek38 39### The Most Advanced Open-Source Language Model for Uzbek40 41[![Model](https://img.shields.io/badge/🤗_Model-NeuronAI--Uzbek-blue)](https://huggingface.co/NeuronUz/NeuronAI-Uzbek)42[![License](https://img.shields.io/badge/License-Apache_2.0-green.svg)](https://opensource.org/licenses/Apache-2.0)43[![Base Model](https://img.shields.io/badge/Base-Qwen3--4B-purple)](https://huggingface.co/Qwen/Qwen3-4B)44 45**🏆 4th Place Globally | 🥇 1st Place in Uzbekistan on UzLiB Benchmark**46 47*Outperforming GPT-4o, Claude 3.5 Sonnet, and Gemini 2.5 Flash on Uzbek language tasks*48 49</div>50 51---52 53## 📊 Key Results54 55<div align="center">56 57| Achievement | Value |58|-------------|-------|59| **UzLiB Overall Score** | **0.662** |60| **Global Ranking** | **#4** |61| **Regional Ranking** | **#1 in Uzbekistan** |62| **Tokenizer Efficiency Improvement** | **+22.5%** vs Qwen3-4B |63 64</div>65 66---67 68## 🏆 UzLiB Benchmark Performance69 70NeuronAI-Uzbek achieves exceptional performance on the [UzLiB Benchmark](https://github.com/tahrirchi/uzlib/blob/main/LEADERBOARD.md), the comprehensive evaluation suite for Uzbek language understanding.71 72### Leaderboard Position73 74[![image](https://cdn-uploads.huggingface.co/production/uploads/65fc70cbaeca3946b8753017/2xJ9BjS6rPNoRoBAzvW7w.png)](https://github.com/tahrirchi/uzlib/blob/main/LEADERBOARD.md)75 76 77> **Note**: NeuronAI-Uzbek is the **smallest model** in the top 10, with only **4B parameters**, while competing against models with 100B+ parameters.78 79### Performance Comparison vs Original Qwen3-4B80 81| Metric | Qwen3-4B (Original) | NeuronAI-Uzbek | Improvement |82|--------|:-------------------:|:--------------:|:-----------:|83| **Overall (All)** | 0.345 | **0.662** | **+91.9%** |84| Correct Word | 0.351 | 0.718 | +104.6% |85| Meaning | 0.309 | 0.466 | +50.8% |86| Meaning in Context | 0.347 | 0.333 | -4.0% |87| Fill-in | 0.327 | 0.385 | +17.7% |88 89---90 91## 🔤 Tokenizer Efficiency92 93We optimized the tokenizer specifically for Uzbek, achieving significantly better tokenization efficiency (lower fertility rate = fewer tokens per word = faster inference and lower costs).94 95### Fertility Rate Comparison96 97| Model | Fertility Rate | Std Dev | Vocab Size | Improvement vs Qwen3 |98|-------|:--------------:|:-------:|:----------:|:--------------------:|99| **NeuronAI-Uzbek (Ours)** 🏆 | **2.67** | 0.15 | 180,000 | **+22.5%** |100| Gemma 2-9B | 3.15 | 0.22 | 256,000 | +8.3% |101| LLaMA 3.1-8B | 3.32 | 0.22 | 128,256 | +3.7% |102| DeepSeek-V3 | 3.32 | 0.21 | 128,815 | +3.4% |103| Qwen3-4B (Original) | 3.44 | 0.22 | 151,669 | - |104 105> **Fertility Rate**: Average number of tokens per word. Lower is better for efficiency.106 107<div align="center">108<img src="assets/fertility_comparison_chart.png" alt="Tokenizer Fertility Rate Comparison" width="700"/>109</div>110 111### What This Means112 113- **22.5% fewer tokens** needed to represent Uzbek text114- **Faster inference** due to shorter sequences115- **Lower API costs** when deployed116- **Better context utilization** - fit more content in the same context window117 118---119 120## 🛠️ Model Details121 122### Architecture123 124| Property | Value |125|----------|-------|126| **Base Model** | Qwen3-4B |127| **Parameters** | 4 Billion |128| **Vocabulary Size** | 180,000 tokens |129| **Context Length** | 32,768 tokens |130| **Architecture** | Transformer (Decoder-only) |131| **Precision** | BFloat16 |132 133### Training Methodology134 1351. **Tokenizer Surgery**: Extended vocabulary with 40,000 Uzbek-optimized tokens1362. **Embedding Initialization**: Semantic initialization using subword composition1373. **Continual Pretraining**: Trained on 2B tokens of Uzbek and English text corpus1384. **Instruction Fine-tuning**: Aligned using Uzbek and English instruction datasets139 140### Training Data141 142| Dataset | Type | Purpose |143|---------|------|---------|144| Uzbek Web Corpus | Pretraining | Language modeling |145| behbudiy/alpaca-cleaned-uz | SFT | Uzbek instructions |146| NeuronUz/uzbek-spelling-mcq | SFT | Benchmark-targeted training |147| vicgalle/alpaca-gpt4 | SFT | English capability retention |148 149---150 151## 🚀 Quick Start152 153### Installation154 155```bash156pip install transformers torch157```158 159### Basic Usage160 161```python162from transformers import AutoModelForCausalLM, AutoTokenizer163 164model_name = "NeuronUz/NeuronAI-Uzbek"165 166tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)167model = AutoModelForCausalLM.from_pretrained(168    model_name,169    torch_dtype="auto",170    device_map="auto",171    trust_remote_code=True172)173 174prompt = "O'zbekiston haqida qisqacha ma'lumot bering."175 176messages = [177    {"role": "user", "content": prompt}178]179 180text = tokenizer.apply_chat_template(181    messages,182    tokenize=False,183    add_generation_prompt=True184)185 186inputs = tokenizer(text, return_tensors="pt").to(model.device)187outputs = model.generate(188    **inputs,189    max_new_tokens=512,190    temperature=0.7,191    top_p=0.9,192    do_sample=True193)194 195response = tokenizer.decode(outputs[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True)196print(response)197```198 199### With Thinking Mode (Chain-of-Thought)200 201```python202messages = [203    {"role": "user", "content": "5 ta 3 ga bo'linuvchi 100 dan kichik natural sonlarni toping."}204]205 206text = tokenizer.apply_chat_template(207    messages,208    tokenize=False,209    add_generation_prompt=True,210    enable_thinking=True  # Enable step-by-step reasoning211)212```213 214---215 216## 📈 Use Cases217 218NeuronAI-Uzbek excels at:219 220- **📝 Text Generation**: Creative writing, content creation in Uzbek221- **❓ Question Answering**: Answering questions about Uzbek culture, history, and general knowledge222- **📚 Reading Comprehension**: Understanding and analyzing Uzbek texts223- **🔤 Grammar & Spelling**: Uzbek language correctness tasks224- **🌐 Translation Assistance**: Uzbek-English language tasks225- **💬 Conversational AI**: Building Uzbek chatbots and assistants226 227---228 229## ⚠️ Limitations230 231- **Knowledge Cutoff**: Training data has a knowledge cutoff date232- **Hallucinations**: May generate plausible-sounding but incorrect information233- **Bias**: May reflect biases present in training data234- **Not for Critical Applications**: Should not be used for medical, legal, or safety-critical applications without human oversight235 236---237 238## 📜 License239 240This model is released under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0).241 242---243 244## 🙏 Acknowledgments245 246- **Qwen Team** at Alibaba for the excellent Qwen3-4B base model247- **UzLiB Benchmark** creators for the comprehensive evaluation framework248- **Uzbek NLP Community** for datasets and linguistic resources249 250---251 252## 📖 Citation253 254```bibtex255@misc{neuronai-uzbek-2025,256  title={NeuronAI-Uzbek: An Optimized Language Model for Uzbek},257  author={NeuronAI Team},258  year={2025},259  publisher={Hugging Face},260  url={https://huggingface.co/NeuronUz/NeuronAI-Uzbek}261}262```263 264---265 266<div align="center">267 268**Built with ❤️ in Uzbekistan by [NeuronUz](https://huggingface.co/NeuronUz)**269 270</div>271