Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch
Paper • 2311.03099 • Published • 32
An attempt to add a bit of depression and med info to Lunaris steering it into a somewhat darker mood and maybe improve its logic slightly. Not sure if it worked though.
This is a merge of pre-trained language models created using mergekit.
This model was merged using the linear DARE merge method using Sao10K/L3-8B-Lunaris-v1 as a base.
The following models were included in the merge:
The following YAML configuration was used to produce this model:
models:
- model: Sao10K/L3-8B-Lunaris-v1
- model: FreedomIntelligence/HuatuoGPT-o1-8B
parameters:
weight: 0.1
- model: Cas-Warehouse/Llama-3-MopeyMule-Blackroot-8B
parameters:
weight: 0.2
- model: Cas-Warehouse/Llama-3-Mopeyfied-Psychology-v2
parameters:
weight: 0.1
merge_method: dare_linear
base_model: Sao10K/L3-8B-Lunaris-v1
dtype: bfloat16