Do InternVL3-1B-i1-GGUF support image to text
#1
by romyull - opened
Do you know how can I use this model for image to text inference?
I want to use this quantized version with llama.cpp
Do I need to have mmproj file for that?
Do you know how can I use this model for image to text inference?
You just download and specify the provided mmproj file when loading the model with llama.cpp
I want to use this quantized version with llama.cpp
Yes you can do so. You can choose between any of the static or weighted/imatrix quants for the LLM layers and either use Q8 or F16 for the vision layers.
Do I need to have mmproj file for that?
Yes you have to use one of the following mmproj files:
Thank you for your reply.
Again, thanks for showing me the mmproj file. I missed it totally.