huihui-ai

/

Huihui-Step3-VL-10B-abliterated

like 19

To save space, you can quantify it yourselves.

I've got too poor hardware to make gguf's.

GGUF quantization does not require GPU memory.

Pleeeeaaaase... :))
We need gguf.

If it's okay I make request of making Gguf's of this model from mradermacher - https://huggingface.co/mradermacher/model_requests/discussions/1793

convert_hf_to_gguf fails with Can not map tensor 'model.vit_large_projector.weight after

INFO:hf-to-gguf:blk.35.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:output_norm.weight,        torch.bfloat16 --> F32, shape = {4096}

convert_hf_to_gguf fails with Can not map tensor 'model.vit_large_projector.weight after

INFO:hf-to-gguf:blk.35.attn_q_norm.weight, torch.bfloat16 --> F32, shape = {128}
INFO:hf-to-gguf:output_norm.weight,        torch.bfloat16 --> F32, shape = {4096}

Hi, come to my repo for GGUF, you could build llama-server or llama-mtmd-cli yourself from my branch: https://github.com/Kraven1109/llama.cpp
it's quite fast on my laptop

Please for Gguf's to work at Lmstudio.

hi

Found it https://huggingface.co/kraven1109/Step3-VL-10B-GGUF-abliterated
Great job!