You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

merged_pg67a

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the SLERP merge method.

Models Merged

The following models were included in the merge:

Configuration

The following YAML configuration was used to produce this model:


merge_method: slerp
dtype: bfloat16
out_dtype: bfloat16

base_model: UmbrellaInc/Prototype-Virus-1B

models:
  - model: UmbrellaInc/Prototype-Virus-1B
    parameters:
      weight: 0.30
  - model: thelamapi/next-1b
    parameters:
      weight: 0.40

parameters:
  t: 0.35
  normalize: false
  rescale: true
  rescale_factor: 1.14
  memory_efficient: true
  low_cpu_mem_usage: true

layer_range:
  - value: [4, 22]

tie_word_embeddings: true
tie_output_embeddings: true

Downloads last month: 3

Safetensors

Model size

1.0B params

Tensor type

BF16

Model tree for UmbrellaInc/PG67A-W-Serum-1B

UmbrellaInc/Prototype-Virus-1B

thelamapi/next-1b

Merge model

this model

Finetunes

2 models

Quantizations

2 models