locailabs/gemma-3-1b-it-sft-metamathqa-modelmerge

Gemma 3 1B IT — MetaMathQA Merged (α=0.5)

A merged model created by interpolating the weights of a MetaMathQA-finetuned Gemma 3 1B IT with the original base model.

Method

1. 1.Fine-tune google/gemma-3-1b-it on 7,000 samples from MetaMathQA using SFT.
2. 2.Merge the fine-tuned weights back into the base model via linear interpolation with α=0.5:

$$\theta{\text{merged}} = \alpha \cdot \theta{\text{FT}} + (1 - \alpha) \cdot \theta_{\text{base}}$$

This simple averaging actually improves task-specific gain from fine-tuning while retaining more of the base model's instruction following that pure FT degrades.

Results