Could you add Vision?
#1
by Ula111 - opened
Could you add Vision file? mmproj?
Other gguf's got it ready. Like:
https://huggingface.co/bartowski/ServiceNow-AI_Apriel-1.6-15b-Thinker-GGUF/blob/main/mmproj-ServiceNow-AI_Apriel-1.6-15b-Thinker-bf16.gguf
As far as I know, Heretic doesn't touch or decensor the vision part / mmproj. So you can use the original mmproj instead.
if i have time i will re-quantize to include the vision part into gguf
I have no idea but maybe if you upload here orginal mmproj file LmStudio will download it automatic like with other models?
I have no idea with lm studio. But with kobold.cpp and llama.cpp we can custom mmproj gguf file instead using original vision from model.
I just download and copy mmproj f16 file to LmStudio Apriel folder and Vision works. Thanks.
MagicalAlchemist changed discussion status to closed