SamsungSAILMontreal/Qwen3-1.7B-Math-Code-Fr

1---2library_name: transformers3tags:4- small-lm5- math6- code7- reasoning8- slm9- french10- merge11license: apache-2.012base_model:13- SamsungSAILMontreal/Qwen3-1.7B14- SamsungSAILMontreal/Qwen3-1.7B-Math15- SamsungSAILMontreal/Qwen3-1.7B-Code16- SamsungSAILMontreal/Qwen3-1.7B-Fr17language:18- fr19- en20---21 22# Qwen3-1.7B-Math-Fr23 24This model is obtained by merging [SamsungSAILMontreal/Qwen3-1.7B-Math](https://huggingface.co/SamsungSAILMontreal/Qwen3-1.7B-Math), 25[SamsungSAILMontreal/Qwen3-1.7B-Code](https://huggingface.co/SamsungSAILMontreal/Qwen3-1.7B-Code) and 26[SamsungSAILMontreal/Qwen3-1.7B-Fr](https://huggingface.co/SamsungSAILMontreal/Qwen3-1.7B-Fr).27The model is used in the experiments described in https://bknyaz.github.io/blog/2026/meta-merge/. 28Single A100 was used for merging and evaluation.29 30The following versions were used for merge/eval:31 32- python >= 3.1033- torch               : 2.9.0+cu12834- lm_eval             : 0.4.9.135- vllm                : 0.11.136- transformers        : 4.57.637- datasets            : 3.2.038- numpy               : 2.2.639 40## Merging41 42Merging was done using parameter averaging implemented in [merge_qwen.py](https://github.com/SamsungSAILMontreal/nino/blob/main/merge_qwen.py).43 44## Evaluation45 46Evaluation was done with lm_eval on the test split of [gsm8k](https://huggingface.co/datasets/openai/gsm8k), 47[french_bench (avg score)](https://github.com/EleutherAI/lm-evaluation-harness/tree/main/lm_eval/tasks/french_bench), 48[gsm8k-fr](https://huggingface.co/datasets/cmh/gsm8k_fr) and humaneval (instruct):49 50```bash51python -m lm_eval --model vllm --model_args pretrained=${model},tensor_parallel_size=1,dtype=auto,gpu_memory_utilization=0.9,data_parallel_size=1 \52 --tasks gsm8k,french_bench,gsm8k-fr,humaneval_instruct --batch_size 1 --apply_chat_template=True --confirm_run_unsafe_code --trust_remote_code53```54 55To evaluate on gsm8k-fr you can use our fork https://github.com/bknyaz/lm-evaluation-harness/tree/main/lm_eval/tasks/gsm8k. 56 57### Results58 59| Model                     | gsm8k | french | gsm8k-fr | humaneval_instruct | avg  |60|---------------------------|-------|--------|----------|--------------------|------|61| Qwen3-1.7B                | 20.6  | 26.2   | 20.2     | 67.1               | 33.5 |62| Qwen3-1.7B-Math           | 62.1  | 28.3   | 41.5     | 48.2               | 45.0 |63| Qwen3-1.7B-Fr             | 60.9  | 32.8   | 43.9     | 56.1               | 48.4 |64| Qwen3-1.7B-Code           | 56.7  | 28.2   | 36.5     | 69.5               | 47.7 |65| Qwen3-1.7B-Math-Code-Fr   | 64.6  | 29.2   | 48.4     | 65.2               | 51.8 |66 67## License68 69Please refer to the license of the base models [SamsungSAILMontreal/Qwen3-1.7B-Math](https://huggingface.co/SamsungSAILMontreal/Qwen3-1.7B-Math), 70[SamsungSAILMontreal/Qwen3-1.7B-Code](https://huggingface.co/SamsungSAILMontreal/Qwen3-1.7B-Code) and 71[SamsungSAILMontreal/Qwen3-1.7B-Fr](https://huggingface.co/SamsungSAILMontreal/Qwen3-1.7B-Fr).