vishinvents/OpenEuroLLM-Czech-vLLM-GGUF

OpenEuroLLM-Czech vLLM GGUF

This repository is a deployment-focused package for serving the Czech-flavored OpenEuroLLM-Czech experience with vLLM.

It uses a vLLM-compatible split Gemma 3 GGUF layout:

• gemma-3-12b-it-Q4_K_M.gguf: language model weights
• mmproj-model-f16.gguf: vision encoder and projector weights

The Czech behavior is provided by the default system prompt in czech-system-prompt.txt, adapted from the original Ollama package jobautomation/OpenEuroLLM-Czech.

Important Notes

• This is not a separately fine-tuned Czech checkpoint.
• The underlying weights are Gemma 3 12B GGUF weights packaged for vLLM deployment.
• To reproduce the Czech assistant behavior, send the system prompt from czech-system-prompt.txt with your requests.

Files

• gemma-3-12b-it-Q4_K_M.gguf
• mmproj-model-f16.gguf
• config.json
• generation_config.json
• czech-system-prompt.txt
• NOTICE
• licenses/

Recommended Deployment

The safest path is to download the repository locally first so vLLM can resolve the sidecar mmproj file in the same directory.

hf download vishinvents/OpenEuroLLM-Czech-vLLM-GGUF --local-dir ./OpenEuroLLM-Czech-vLLM-GGUF

Then serve the local GGUF file:

vllm serve ./OpenEuroLLM-Czech-vLLM-GGUF/gemma-3-12b-it-Q4_K_M.gguf \
  --tokenizer unsloth/gemma-3-12b-it \
  --served-model-name OpenEuroLLM-Czech

If you are using images, keep mmproj-model-f16.gguf in the same directory as the main GGUF.

Using the Czech Prompt

Send the contents of czech-system-prompt.txt as the system message in your chat requests.

License

This repository includes the license materials needed for redistribution:

• licenses/Gemma-Terms-of-Use.txt
• licenses/Additional-License.txt

See NOTICE for the Gemma redistribution notice.