virtuous7373/Lambent-Mira-Testing-Ground-27B

All credits go to Lambent for the wonderful model series.

Lambent-Mira-M1-27B-27B-Q5_K_X.gguf | Q5_K_X | 18.4GiB | Final estimate: PPL = 6.5653 +/- 0.04305 |

• Turtle upload speed, they will pop up, eventually.

This is a merge of pre-trained language models created using mergekit.

Merge Method

This model was merged using the Karcher, CABS, and Della,

Configuration

The following YAML configurations were used to produce this model:

Lambent Mira-M1

# https://huggingface.co/Lambent
name: Mira M1
models:
    - model: ./Ingredients/Lambent_Mira-v1.17-27B-Custom-Heretic
    - model: ./Ingredients/Lambent_Mira-v1-dpo-27B
    - model: ./Ingredients/Lambent_Mira-1.10-dpo-27B
    - model: ./Ingredients/Lambent_Mira-v1.8.1a-27B
    - model: ./Ingredients/Lambent_Mira-v1.12-Ties-27B
    - model: ./Ingredients/Lambent_Mira-v1.3-27B
    - model: ./Ingredients/Lambent_Mira-v1.5-27B
merge_method: karcher
parameters:
    max_iter: 1000
    tol: 1e-9
normalize: false
int8_mask: true
tokenizer:
    source: union
dtype: bfloat16
out_dtype: bfloat16

Lambent Mira-M2

# https://huggingface.co/Lambent
name: Mira M2
models:
    - model: ./Ingredients/Lambent_Mira-v1.8.1a-27B
    - model: ./Ingredients/Lambent_Mira-v1.17-27B-Custom-Heretic
      parameters:
          weight: 0.4
          n_val: 16
          m_val: 32
    - model: ./Ingredients/mira-m1
      parameters:
          weight: 0.2
          n_val: 12
          m_val: 32
merge_method: cabs
default_n_val: 8
default_m_val: 32
pruning_order:
    - ./Ingredients/Lambent_Mira-v1.17-27B-Custom-Heretic
    - ./Ingredients/mira-m1
base_model: ./Ingredients/Lambent_Mira-v1.8.1a-27B
dtype: float32
out_dtype: bfloat16
tokenizer:
    source: union

Lambent Mira-M3

# https://huggingface.co/Lambent
name: Mira M3
models:
    - model: ./Ingredients/Lambent_Mira-1.10-dpo-27B
    - model: ./Ingredients/mira-m2
      parameters:
          weight: 0.4
          n_val: 16
          m_val: 32
    - model: ./Ingredients/mira-m1
      parameters:
          weight: 0.2
          n_val: 12
          m_val: 32
merge_method: cabs
default_n_val: 8
default_m_val: 32
pruning_order:
    - ./Ingredients/mira-m2
    - ./Ingredients/mira-m1
base_model: ./Ingredients/Lambent_Mira-1.10-dpo-27B
dtype: float32
out_dtype: bfloat16
tokenizer:
    source: union

Lambent Mira-M4

name: Mira M4
models:
    - model: ./Ingredients/Mira-solid-base

    - model: ./Ingredients/mira-m1
      parameters:
          weight: 0.55
          density: 0.5
          epsilon: 0.4
    - model: ./Ingredients/mira-m2
      parameters:
          weight: 0.25
          density: 0.5
          epsilon: 0.4
    - model: ./Ingredients/mira-m3
      parameters:
          weight: 0.25
          density: 0.4
          epsilon: 0.3
merge_method: della
base_model: ./Ingredients/Mira-solid-base
parameters:
    lambda: 0.9
    normalize: true
dtype: bfloat16
tokenizer:
    source: base

Quant Recipe

# Quant scheme inspired by: https://huggingface.co/ddh0/Q4_K_X.gguf
llama_quant="$LLAMAQUANT"
imatrix="$KITCHEN/Bartowski-Gemma-3-27B-imatrix.gguf" # Thanks =)
model=$KITCHEN/Models_cooking/Mira-M1-00001-of-00003.gguf
outpath="=$KITCHEN/Models/Mira-M1-27B-Q5_K_X.gguf"
quant_type="Q8_0"

blk_all="blk\.(0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32|33|34|35|36|37|38|39|40|41|42|43|44|45|46|47|48|49|50|51|52|53|54|55|56|57|58|59|60|61)"
blk_step="blk\.(0|1|2|3|4|5|6|9|12|15|18|21|24|27|30|33|36|39|42|45|48|51|54|55|56|57|58|59|60|61)"
blk_alt="blk\.(7|8|10|11|13|14|16|17|19|20|22|23|25|26|28|29|31|32|34|35|37|38|40|41|43|44|46|47|49|50|52|53)"

custom=(
    --tensor-type token_embd.weight=Q5_K
    --tensor-type "${blk_all}\.attn_k.weight=Q8_0"
    --tensor-type "${blk_all}\.attn_output.weight=Q6_K"
    --tensor-type "${blk_all}\.attn_q.weight=Q5_K"
    --tensor-type "${blk_step}\.attn_v.weight=Q8_0" # eh, let her fly
    --tensor-type "${blk_step}\.ffn_down.weight=Q6_K"
    --tensor-type "${blk_all}\.ffn_gate.weight=Q5_K"
    --tensor-type "${blk_all}\.ffn_up.weight=Q5_K"
    --tensor-type "${blk_alt}\.attn_v.weight=Q8_0"
    --tensor-type "${blk_alt}\.ffn_down.weight=Q5_K"
)

"$llama_quant" --imatrix "$imatrix" "${custom[@]}" "$model" "$outpath" "$quant_type"

Nice people:

Lambent - For creating the Mira series.

Bartowski - For the Imatrix, work with Arcee, and overall pilar to community.

ddho - For the quant scheme idea. (https://huggingface.co/ddh0/Q4_K_X.gguf)

ubergarm - For all your public guides, from quanting to perplexity, overall positivity, and making things that appear intimidating much more approachable. I would not have been doing any of this if it weren't for you. Thank you.

win10 - For making Karcher, awesome work and method.