perf(deploy): keep model resident + pre-warm to eliminate cold reload

Diagnosis from Space runtime logs:
- GPU works; Gemma 4 E4B q4_K_M fully offloads 43/43 layers to T4.
- Inference is 47 tok/s once hot — fine.
- Cold model load (mmap 9GB from /data persistent disk into VRAM) is
~150s. Ollama's default keep_alive=5m was evicting between user calls,
so every extraction in a demo session re-paid the load cost (~200s
end-to-end).

Fix:
- OLLAMA_KEEP_ALIVE=24h via Dockerfile ENV: model stays in VRAM as long
as the container is awake. Subsequent extractions land hot (~14s).
- Pre-warm with a one-token generate at the end of entrypoint.sh so the
first user request also lands hot. Adds ~2-3 min to boot, but the
Space's app_port doesn't accept traffic until uvicorn comes up
anyway, so the user-facing boot time is the same.

Wake-from-sleep (HF gcTimeout, default 15min idle) still pays the cold
load — unavoidable on a serverless GPU host. Mitigations for the demo
are at Space-settings level (raise sleep timeout) or workflow level
(warm the Space before showing it to a judge).

Dockerfile

@@ -74,7 +74,8 @@ COPY --from=frontend-builder /build/frontend/dist ./frontend/dist
 ENV PORT=7860 \
     OLLAMA_MODEL=gemma4:e4b-it-q4_K_M \
     OLLAMA_MODELS=/data/.ollama/models \
-    HF_HOME=/data/.cache/huggingface
+    HF_HOME=/data/.cache/huggingface \
+    OLLAMA_KEEP_ALIVE=24h
 
 EXPOSE 7860
 

entrypoint.sh

@@ -60,10 +60,23 @@ done
 if ollama list | awk '{print $1}' | grep -qx "$MODEL"; then
   echo "[entrypoint] Model $MODEL already present, skipping pull"
 else
-  echo "[entrypoint] Pulling $MODEL (first boot only — ~4GB, takes 2-5 min)..."
+  echo "[entrypoint] Pulling $MODEL (first boot only — ~9GB, takes 5-15 min)..."
   ollama pull "$MODEL"
 fi
 
+# Pre-warm the model into VRAM so the first user request doesn't pay the
+# ~150s mmap cost (T4 + 9GB weights on HF persistent disk). Combined with
+# OLLAMA_KEEP_ALIVE=24h in Dockerfile ENV, the model stays resident for the
+# life of the container.
+echo "[entrypoint] Pre-warming $MODEL into VRAM (one-shot generate, ~2-3 min on cold persistent disk)..."
+WARM_START=$(date +%s)
+curl -fsS http://127.0.0.1:11434/api/generate \
+  -H "Content-Type: application/json" \
+  -d "{\"model\":\"$MODEL\",\"prompt\":\"ok\",\"stream\":false,\"keep_alive\":\"24h\"}" \
+  >/tmp/warmup.json 2>&1 \
+  && echo "[entrypoint] Pre-warm done in $(($(date +%s) - WARM_START))s" \
+  || echo "[entrypoint] WARN: pre-warm failed — first user request will pay cold-load cost. See /tmp/warmup.json"
+
 # Hand off to FastAPI. uvicorn imports api:app, which imports app.py (loads
 # schemas eagerly via the FastAPI startup hook). Whisper model is loaded
 # lazily on the first audio request — keeps boot fast.