CloudGoat/Qwen3.5-0.8B-JP-Tuned-v2.0

Qwen3.5-0.8B-JP-Tuned-v2.0

This model was created by merging two different Japanese-tuned Qwen3.5-0.8B models and the base model using DARE-TIES. The merge was performed in a way that avoids attention conflicts between the models. I used mergekit.

Benchmark

• —JCommonsenseQA: 67.11%
• —JNLI: 40.30%
• —JSQuAD: 47.68%
• —MARC-ja: 90.57%

Overall, performance improved by 6.02% compared to the base model and by 2.49% compared to v1.0, with notable gains observed in MARC-ja. The initial objective of specializing in "everyday conversation" has been achieved. However, the decline in scores for JNLI and JSQuAD indicates a reduction in logical reasoning capabilities and accuracy when handling long texts.

Merge Details

Merge Method

This model was merged using the DARE TIES merge method using Qwen/Qwen3.5-0.8B as a base.

Models Merged

The following models were included in the merge:

• —Holy-fox/Qwen3.5-0.8B-JP
• —CloudGoat/Qwen3.5-0.8B-JP-Tuned-v1.1
• —Qwen/Qwen3.5-0.8B

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: Holy-fox/Qwen3.5-0.8B-JP                  
      density: 0.80                                 
      weight: [0.5, 0.2, 0.4]                        
  - model: CloudGoat/Qwen3.5-0.8B-JP-Tuned-v1.1      
    parameters:
      density: 0.80
      weight: [0.5, 0.8, 0.6]                       
merge_method: dare_ties
base_model: Qwen/Qwen3.5-0.8B
parameters:
  int8_mask: true
  normalize: true                                    
dtype: bfloat16

Thanks

I would like to express my sincere gratitude to the following organizations for their assistance in creating this model.

• —Unsloth
• —Qwen (Tongyi lab)
• —理化学研究所（理研） 革新知能統合研究センター（AIP） 言語情報アクセス技術チーム
• —Arcee-Ai
• —Holy-fox