when gguf???
#5
by VivekLeon - opened
when gguf???
Hi @VivekLeon ! llama.cpp and GGUF support is definitely on the TODO list. Since this model uses an adapter to conditionally support vision inputs, there will be some architectural changes needed in the mtmd layer of llama.cpp so that the adapter is only activated when image inputs are present. This will also mean that other engines such as Ollama may need similar changes, so the rollout of GGUF support will be a bit slower than some other model architectures unfortunately.
agree it could be awesome!!