darkc0de

/

Mistral-Small-4-119B-2603-heretic

like 4

Is it possible to create ggufs of this model? I've tried with llama.cpp, but it doesn't work, it returns this error python convert_hf_to_gguf.py
/mnt/shared/LM_Studio_Models/Mistral-Small-4-119B-2603-heretic
--outtype f16
--outfile /mnt/shared/LM_Studio_Models/Mistral-Small-4-119B-2603-heretic/Mistral-Small-4-119B-2603-heretic.gguf
INFO:hf-to-gguf:Loading model: Mistral-Small-4-119B-2603-heretic
INFO:hf-to-gguf:Model architecture: Mistral3ForConditionalGeneration
INFO:hf-to-gguf:gguf: loading model weight map from 'model.safetensors.index.json'
INFO:hf-to-gguf:gguf: indexing model part 'model-00001-of-00006.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'model-00002-of-00006.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'model-00003-of-00006.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'model-00004-of-00006.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'model-00005-of-00006.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'model-00006-of-00006.safetensors'
INFO:hf-to-gguf:heuristics detected bfloat16 tensor dtype, setting --outtype bf16
INFO:gguf.gguf_writer:gguf: This GGUF file is for Little Endian only
INFO:hf-to-gguf:gguf: loading model weight map from 'model.safetensors.index.json'
INFO:hf-to-gguf:gguf: indexing model part 'model-00001-of-00006.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'model-00002-of-00006.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'model-00003-of-00006.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'model-00004-of-00006.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'model-00005-of-00006.safetensors'
INFO:hf-to-gguf:gguf: indexing model part 'model-00006-of-00006.safetensors'
INFO:hf-to-gguf:heuristics detected bfloat16 tensor dtype, setting --outtype bf16
INFO:gguf.gguf_writer:gguf: This GGUF file is for Little Endian only
INFO:hf-to-gguf:Exporting model...
INFO:hf-to-gguf:output.weight, torch.bfloat16 --> BF16, shape = {4096, 131072}
Traceback (most recent call last):
File "/home/juan/llama.cpp/convert_hf_to_gguf.py", line 12820, in
main()
~~~~^^
File "/home/juan/llama.cpp/convert_hf_to_gguf.py", line 12814, in main
model_instance.write()
~~~~~~~~~~~~~~~~~~~~^^
File "/home/juan/llama.cpp/convert_hf_to_gguf.py", line 8800, in write
self.impl.write()
~~~~~~~~~~~~~~~^^
File "/home/juan/llama.cpp/convert_hf_to_gguf.py", line 934, in write
self.prepare_tensors()
~~~~~~~~~~~~~~~~~~~~^^
File "/home/juan/llama.cpp/convert_hf_to_gguf.py", line 8731, in prepare_tensors
super().prepare_tensors()
~~~~~~~~~~~~~~~~~~~~~~~^^
File "/home/juan/llama.cpp/convert_hf_to_gguf.py", line 794, in prepare_tensors
for new_name, data_torch in (self.modify_tensors(data_torch, name, bid)):
~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
File "/home/juan/llama.cpp/convert_hf_to_gguf.py", line 8772, in modify_tensors
yield from super().modify_tensors(data_torch, name, bid)
File "/home/juan/llama.cpp/convert_hf_to_gguf.py", line 8728, in modify_tensors
yield from super().modify_tensors(data_torch, name, bid)
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
File "/home/juan/llama.cpp/convert_hf_to_gguf.py", line 545, in modify_tensors
new_name = self.map_tensor_name(name)
File "/home/juan/llama.cpp/convert_hf_to_gguf.py", line 538, in map_tensor_name
raise ValueError(f"Can not map tensor {name!r}")
ValueError: Can not map tensor 'model.language_model.model.embed_tokens.weight'
--outtype: no se encontró la orden