SmolVLM-500M-Instruct-GGUF (Q4_K_M)

Q4_K_M quantized GGUF conversion of SmolVLM-500M-Instruct.

Files

File	Size	Description
SmolVLM-500M-Instruct-Q4_K_M.gguf	289 MB	Main model (Q4_K_M quantized)
mmproj-SmolVLM-500M-Instruct-f16.gguf	190 MB	Vision projector (F16)

Usage

With Ollama

FROM ./SmolVLM-500M-Instruct-Q4_K_M.gguf
PROJECTOR ./mmproj-SmolVLM-500M-Instruct-f16.gguf
PARAMETER num_ctx 4096
PARAMETER temperature 0.1

With llama.cpp

./llama-mtmd-cli -m SmolVLM-500M-Instruct-Q4_K_M.gguf --mmproj mmproj-SmolVLM-500M-Instruct-f16.gguf --image screenshot.png -p "Click the Submit button"

License

Apache 2.0