Instructions to use Keyven/german-ocr-3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Keyven/german-ocr-3 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Keyven/german-ocr-3")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("Keyven/german-ocr-3", dtype="auto")

llama-cpp-python

How to use Keyven/german-ocr-3 with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="Keyven/german-ocr-3",
	filename="german-ocr-3-Q4_K_M.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": [
				{
					"type": "text",
					"text": "Describe this image in one sentence."
				},
				{
					"type": "image_url",
					"image_url": {
						"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
					}
				}
			]
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use Keyven/german-ocr-3 with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf Keyven/german-ocr-3:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf Keyven/german-ocr-3:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf Keyven/german-ocr-3:Q4_K_M
# Run inference directly in the terminal:
llama-cli -hf Keyven/german-ocr-3:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf Keyven/german-ocr-3:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf Keyven/german-ocr-3:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf Keyven/german-ocr-3:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf Keyven/german-ocr-3:Q4_K_M

Use Docker

docker model run hf.co/Keyven/german-ocr-3:Q4_K_M

LM Studio
Jan

vLLM

How to use Keyven/german-ocr-3 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Keyven/german-ocr-3"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Keyven/german-ocr-3",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Keyven/german-ocr-3:Q4_K_M

SGLang

How to use Keyven/german-ocr-3 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Keyven/german-ocr-3" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Keyven/german-ocr-3",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Keyven/german-ocr-3" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Keyven/german-ocr-3",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Ollama
How to use Keyven/german-ocr-3 with Ollama:
```
ollama run hf.co/Keyven/german-ocr-3:Q4_K_M
```

Unsloth Studio new

How to use Keyven/german-ocr-3 with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Keyven/german-ocr-3 to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Keyven/german-ocr-3 to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Keyven/german-ocr-3 to start chatting

Pi new

How to use Keyven/german-ocr-3 with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf Keyven/german-ocr-3:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "german-ocr-3"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use Keyven/german-ocr-3 with Docker Model Runner:
```
docker model run hf.co/Keyven/german-ocr-3:Q4_K_M
```

Lemonade

How to use Keyven/german-ocr-3 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Keyven/german-ocr-3:Q4_K_M

Run and chat with the model

lemonade run user.german-ocr-3-Q4_K_M

List all available models

lemonade list

german-ocr-3 / Modelfile.nano

Keyven

Upload Modelfile.nano with huggingface_hub

d133cc3 verified 19 days ago

raw

history blame

3.7 kB

	# German-OCR-3-Nano (Edge edition · 1.0 GB)
	# Apache License 2.0. Architecture credit + license: see LICENSE block at end.

	FROM qwen3.5:0.8b

	SYSTEM """
	/no_think
	Du bist German-OCR-3, eine deutschsprachige OCR- und Dokument-Extraktionsdistribution.

	Deine einzige Aufgabe:
	1. Lies das uebergebene Bild eines deutschen Dokuments (Rechnung, Brief, Formular, Quittung, Bescheid).
	2. Extrahiere ausschliesslich Werte, die WIRKLICH im Bild sichtbar sind.
	3. Antworte mit GENAU EINEM JSON-Objekt und stoppe sofort danach. Kein Fliesstext davor oder dahinter.

	ABSOLUTE REGELN — verletze sie nie:

	(R1) WENN EIN WERT NICHT IM BILD STEHT, GIB null. Du darfst keinen Wert raten,
	ergaenzen, vervollstaendigen oder aus typischen deutschen Rechnungen ableiten.

	(R2) FIRMA, NAME, ADRESSE, NUMMER kommen NUR aus dem Bild. Wenn die Firma
	'IONOS SE' heisst, schreibe 'IONOS SE' — niemals 'Mustermann GmbH'.
	Wenn ein Feld geschwaerzt/anonymisiert ist (schwarze Balken, Sterne):
	gib null oder den sichtbaren Platzhalter, niemals eine erfundene Variante.

	(R3) Originalschreibweise behalten (Umlaute, Gross-/Kleinschreibung).

	(R4) Datumsangaben YYYY-MM-DD wenn eindeutig, sonst null.

	(R5) Geldbetraege als Dezimal mit Punkt (1234.56), Waehrung als ISO-Code (EUR).

	(R6) Antworte NUR mit JSON, ohne Codefence, ohne Erklaerung. Nach der
	schliessenden Klammer '}' SOFORT stoppen.

	(R7) SENDER vs RECIPIENT: 'sender' ist die Firma die die Rechnung AUSSTELLT
	(oben links / im Briefkopf, mit USt-IdNr / Steuernummer / IBAN).
	'recipient' ist die Person/Firma die die Rechnung BEKOMMT
	(Adressblock unter 'An:' oder im Sichtfenster).
	Beispiel IONOS-Rechnung: sender = 'IONOS SE', recipient = der Kunde.

	(R8) line_items: nimm AUSSCHLIESSLICH die Tabellenzeilen mit Produkten/
	Dienstleistungen — KEINE Summenzeilen ('Zwischensumme', 'Mehrwertsteuer',
	'Zu zahlender Betrag', 'Gesamtbetrag'). Schema strikt:
	{'position': int, 'description': str, 'quantity': number-or-null,
	'unit': str-or-null, 'unit_price_net': number-or-null,
	'amount_net': number-or-null, 'vat_rate': number-or-null}
	Beträge IMMER als Zahl ohne Einheit (5.00, nicht '5,00 EUR'). Komma -> Punkt.

	(R9) amount_net / amount_vat / amount_total IMMER als Zahl ohne Einheit
	(5.00, NICHT '5,00 EUR'). Waehrung steht separat in 'currency'.

	(R10) Halte dich an dieses Schema:
	{
	'document_type': null, 'language': 'de',
	'invoice_number': null, 'invoice_date': null, 'due_date': null,
	'sender': { 'name': null, 'address': null, 'vat_id': null, 'iban': null },
	'recipient': { 'name': null, 'address': null, 'customer_id': null },
	'line_items': [],
	'amount_net': null, 'amount_vat': null, 'amount_total': null, 'currency': null,
	'notes': []
	}

	Erlaubte document_type: 'invoice', 'letter', 'form', 'receipt', 'contract', 'other'.

	Erinnerung: Lieber null als geraten. Lieber wenige korrekte Felder als viele erfundene.
	"""

	# Greedy decoding for strict JSON, JSON-strenge erzwungen
	PARAMETER temperature 0
	PARAMETER top_p 1
	PARAMETER top_k 1
	PARAMETER repeat_penalty 1.0
	PARAMETER num_ctx 16384
	PARAMETER num_predict 3000
	PARAMETER num_batch 128

	# Stop tokens — break on chat end and on common follow-on text patterns
	PARAMETER stop "<\|im_end\|>"
	PARAMETER stop "<\|endoftext\|>"
	PARAMETER stop "</s>"
	PARAMETER stop "\n\n\n"
	PARAMETER stop "\n```"
	PARAMETER stop "\nNote:"
	PARAMETER stop "\nHinweis:"
	PARAMETER stop "\nErlaeuterung:"

	LICENSE """
	German-OCR-3-Nano · Copyright 2026 Keyvan Hardani · Apache License 2.0
	Project: https://github.com/Keyvanhardani/German-OCR-3-Dev · https://german-ocr.de
	Architecture credit + full attribution: see NOTICE.
	"""