Resolving Interference When Merging Models
Paper • 2306.01708 • Published • 18
I tried to make it more humane, creative, and suitable for writing, and it seemed to turn out not bad, but it needs testing.
We need to add Enable deep thinking subroutine to the beginning of the system prompt, and <think> at the start of the sentence.
This is very, very suck, don't download this.
This is a merge of pre-trained language models created using mergekit.
This model was merged using the TIES merge method using ./amoral-cogito-14b as a base.
The following models were included in the merge:
The following YAML configuration was used to produce this model:
name: amoral-cogito-Zara
models:
- model: ./amoral-cogito-14b
parameters:
weight: 0.75
density: 0.6
- model: ./Zara-14b-v1.2
parameters:
weight: 0.25
merge_method: ties
dtype: bfloat16
base_model: ./amoral-cogito-14b