EndLessTime/fine_tuned_per_domain_balanced_moe_c10

1---2library_name: transformers3license: other4base_model: Qwen/Qwen1.5-MoE-A2.7B5tags:6- generated_from_trainer7metrics:8- accuracy9model-index:10- name: fine_tuned_per_domain_balanced_moe_c1011  results: []12---13 14<!-- This model card has been generated automatically according to the information the Trainer had access to. You15should probably proofread and complete it, then remove this comment. -->16 17# fine_tuned_per_domain_balanced_moe_c1018 19This model is a fine-tuned version of [Qwen/Qwen1.5-MoE-A2.7B](https://huggingface.co/Qwen/Qwen1.5-MoE-A2.7B) on the None dataset.20It achieves the following results on the evaluation set:21- Loss: 2.214922- Accuracy: 0.537423 24## Model description25 26More information needed27 28## Intended uses & limitations29 30More information needed31 32## Training and evaluation data33 34More information needed35 36## Training procedure37 38### Training hyperparameters39 40The following hyperparameters were used during training:41- learning_rate: 2e-0542- train_batch_size: 143- eval_batch_size: 144- seed: 4245- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments46- lr_scheduler_type: linear47- num_epochs: 348 49### Training results50 51| Training Loss | Epoch  | Step | Accuracy | Validation Loss |52|:-------------:|:------:|:----:|:--------:|:---------------:|53| 7.9537        | 0.0006 | 100  | 0.5384   | 4.2406          |54| 2.7142        | 0.0013 | 200  | 0.5386   | 6.1312          |55| 2.5969        | 0.0019 | 300  | 0.4651   | 1.0811          |56| 3.6087        | 0.0025 | 400  | 0.4655   | 1.7135          |57| 3.217         | 0.0032 | 500  | 0.5386   | 2.4567          |58| 2.0844        | 0.0038 | 600  | 0.4614   | 3.8137          |59| 3.0955        | 0.0044 | 700  | 0.5386   | 1.2668          |60| 2.0157        | 0.0051 | 800  | 0.5386   | 3.2796          |61| 2.4513        | 0.0057 | 900  | 0.4614   | 2.2765          |62| 2.482         | 0.0063 | 1000 | 0.5386   | 0.7492          |63| 2.3079        | 0.0070 | 1100 | 0.5386   | 1.6933          |64| 2.5698        | 0.0076 | 1200 | 0.5386   | 3.1721          |65| 2.4214        | 0.0082 | 1300 | 0.5386   | 1.7702          |66| 1.2708        | 0.0089 | 1400 | 0.4646   | 0.9111          |67| 0.8665        | 0.0095 | 1500 | 0.5494   | 0.6819          |68| 1.7844        | 0.0101 | 1600 | 0.5386   | 1.7757          |69| 2.9675        | 0.0108 | 1700 | 0.5386   | 2.7387          |70| 2.7119        | 0.0114 | 1800 | 0.5386   | 2.6287          |71| 2.526         | 0.0120 | 1900 | 0.5386   | 1.4967          |72| 3.2745        | 0.0127 | 2000 | 0.4614   | 4.2874          |73| 3.4052        | 0.0133 | 2100 | 1.0082   | 0.4624          |74| 1.7179        | 0.0139 | 2200 | 1.6046   | 0.4666          |75| 2.7225        | 0.0146 | 2300 | 3.3510   | 0.5376          |76| 2.2919        | 0.0152 | 2400 | 3.3149   | 0.5376          |77| 1.729         | 0.0158 | 2500 | 2.1687   | 0.5376          |78| 2.5072        | 0.0165 | 2600 | 2.9068   | 0.5376          |79| 1.9138        | 0.0171 | 2700 | 1.4200   | 0.4624          |80| 1.4881        | 0.0177 | 2800 | 2.2129   | 0.4631          |81| 2.031         | 0.0184 | 2900 | 2.2580   | 0.5370          |82| 1.998         | 0.0190 | 3000 | 2.2149   | 0.5374          |83 84 85### Framework versions86 87- Transformers 4.49.088- Pytorch 2.6.0+cu12689- Datasets 3.3.290- Tokenizers 0.21.091