madDegen commited on
Commit
8a35e8e
·
verified ·
1 Parent(s): fdb2c4a

feat: persistent disk wired + passthrough v3.1

Browse files

GGUF_DIR routes to /data/gguf when HF Pro persistent disk is mounted. Passthrough mode only: qwen3:1.7b triage + nomic-embed-text.

Files changed (1) hide show
  1. startup.sh +18 -0
startup.sh CHANGED
@@ -22,6 +22,24 @@ echo "║ LiteLLM :7860 (Claude-compat proxy) ║"
22
  echo "╚═════════════════════════════════════════════╝"
23
  echo ""
24
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
25
  # ── 1. Start Ollama ───────────────────────────────────────────
26
  export OLLAMA_HOST=0.0.0.0:11434
27
  export OLLAMA_KEEP_ALIVE=5m
 
22
  echo "╚═════════════════════════════════════════════╝"
23
  echo ""
24
 
25
+ # ── Persistent disk (HF Pro) ──────────────────────────────────
26
+ # HF Pro persistent storage mounts at /data
27
+ # GGUF models cached here survive Space restarts (no re-download)
28
+ PERSISTENT_DISK="/data"
29
+ GGUF_DIR="$PERSISTENT_DISK/gguf"
30
+ OLLAMA_MODELS_DIR="$PERSISTENT_DISK/ollama-models"
31
+
32
+ if [ -d "$PERSISTENT_DISK" ]; then
33
+ echo "✅ Persistent disk detected at /data"
34
+ mkdir -p "$GGUF_DIR" "$OLLAMA_MODELS_DIR"
35
+ export OLLAMA_MODELS="$OLLAMA_MODELS_DIR"
36
+ else
37
+ echo "⚠️ No persistent disk — using ephemeral storage"
38
+ echo " Enable in: Space Settings → Persistent Storage → mount /data"
39
+ GGUF_DIR="$HOME/.ollama/gguf"
40
+ mkdir -p "$GGUF_DIR"
41
+ fi
42
+
43
  # ── 1. Start Ollama ───────────────────────────────────────────
44
  export OLLAMA_HOST=0.0.0.0:11434
45
  export OLLAMA_KEEP_ALIVE=5m