locailabs/gemma-3-1b-it-sft-metamathqa-modelmerge

Gemma 3 1B IT — MetaMathQA Merged (α=0.5)

A merged model created by interpolating the weights of a MetaMathQA-finetuned Gemma 3 1B IT with the original base model.

Method

1. Fine-tune google/gemma-3-1b-it on 7,000 samples from MetaMathQA using SFT.
2. Merge the fine-tuned weights back into the base model via linear interpolation with α=0.5:

$${\theta }_{\text{merged}}=\alpha \cdot {\theta }_{\text{FT}}+(1-\alpha )\cdot {\theta }_{\text{base}}\backslash theta\_\{\backslash text\{merged\}\}\; =\; \backslash alpha\; \backslash cdot\; \backslash theta\_\{\backslash text\{FT\}\}\; +\; (1\; -\; \backslash alpha)\; \backslash cdot\; \backslash theta\_\{\backslash text\{base\}\}$$

This simple averaging actually improves task-specific gain from fine-tuning while retaining more of the base model's instruction following that pure FT degrades.

Results

Method	MMLU Redux	GSM8K	IFEval
Base	39.79	33.66	40.48
FT	41.02	37.15	28.84
Merged	40.53	39.58	36.41