docs/future_improvements.md

# Future improvements

A backlog of optimizations that aren't blocking but would tighten the deploy.
None of these are required for current functionality. Order is rough priority,
not commitment.

## Spaces / preload

### ~~0. Re-enable `preload_from_hub` via runtime cache mirror~~ — DONE 2026-05-02

Initial preload deployment failed because HF's build pipeline writes
`~/.cache/huggingface/` as the build user, leaving it read-only for runtime
user 1000. Lazy `hf_hub_download` for non-preloaded files (GGUF, camera LoRAs)
failed with `Permission denied (os error 13)`. `chmod` couldn't help — we
don't own the inode.

Fix landed in `_bootstrap()`'s `_mirror_preload_hf_cache()`:
- Walks `~/.cache/huggingface/` to a parallel `~/hf-cache-rw/` we own
- Hardlinks `blobs/<sha>` files (zero-copy, shared inode, instant reads)
- Preserves relative snapshot symlinks (resolve within the mirror tree)
- Byte-copies `refs/<branch>` files (HF lib overwrites these on etag check)
- Sets `HF_HOME` + `HF_HUB_CACHE` to the mirror so HF lib uses our writable copy
- Falls back to symlink if `os.link()` returns EXDEV (cross-device)

Result: preloaded files are instantly available (cache hit on first generate),
non-preloaded files lazy-download into dirs we own (no permission errors).

### ~~1. Stop preloading models that aren't referenced by any workflow~~ — DONE 2026-05-02

Audit on 2026-05-02 showed two `Lightricks/LTX-2.3` files in `preload_from_hub`
that aren't actually referenced by any workflow JSON we ship:

- `ltx-2.3-22b-dev.safetensors` (~42 GB)
- `ltx-2.3-22b-distilled.safetensors` (~42 GB)

The active path uses `Kijai/LTX2.3_comfy ltx-2.3-22b-dev_transformer_only_bf16.safetensors`.
Removed both — ~84 GB saved. Forced by HF eviction with `storage limit
exceeded (150G)` when total preload was ~234 GB. Risk: if a future workflow
update reintroduces the Lightricks-side filenames, lazy download takes over.

### ~~2. Drop `unsloth/LTX-2.3-GGUF` from preload (~39 GB)~~ — DONE 2026-05-02

Removed alongside (1). GGUF transformer is the low-VRAM alternative; ZeroGPU
H200 has 70 GB so the BF16 transformer always fits. Lazy-loads on first use
of any preset that wires the GGUF path.

### 3. Drop the `Lightricks/LTX-2-19b-LoRA-Camera-Control-Static/Jib-Up/Jib-Down` preload

Each is ~2 GB. The Power Lora Loader has them all listed but defaults all to
`on: false`, so they only load when the user picks one. Lazy-load is
appropriate. Currently kept in preload because of the 10-entry cap +
"easier to keep what we had".

### 4. Auto-generate `preload_from_hub` from `MODEL_REGISTRY`

Today the README list and `MODEL_REGISTRY` in `models.py` can drift. Build a
small `tools/sync_preload.py` that:

1. Reads `MODEL_REGISTRY`
2. Walks the workflow JSONs to find which entries are actually referenced
3. Sorts referenced entries by size (using `huggingface_hub` `repo_info`)
4. Picks the top N entries that fit in the 10-cap
5. Writes them back into the README YAML

Run as a pre-commit or CI step.

### 5. Bake custom-node clones into the build via `requirements.txt` git installs

We currently `git clone` 10 custom-node repos in `_bootstrap()` at runtime.
That's ~30 s of cold start. Some custom nodes ship as pip-installable; for
the others, we could write a small `tools/install_custom_nodes.py` that
runs at build time (via `pip install --no-deps` against git URLs) so the
repos land in the image instead of being fetched at boot.

Tradeoff: Spaces' build pipeline runs the gradio SDK Dockerfile which we
don't control directly. The custom-node clone has to happen at runtime
unless we can move it into the standard `requirements.txt` build step.

### 6. Persistent storage add-on as the "$25/mo button"

If iteration speed becomes the binding constraint, the persistent storage
add-on (Spaces > Settings) at $25/mo for 150 GB makes everything just work
— `/data` is writable, models live there forever, no preload dance.
Sketched approach: `HF_HOME=/data/hf-cache` env var + `_bootstrap()` mkdir
fallback. One-line code change.

## Workflow / runtime

### 7. Move ComfyUI custom-node `requirements.txt` install to build time

Bootstrap currently `pip install`s each custom node's requirements at
runtime. Most are no-ops (deps already in our top-level `requirements.txt`)
but the `pip install --quiet` calls still take a few seconds each. Could
audit and just merge them into the top-level `requirements.txt`.

### 8. Clean up `nodes_replacements.py` warning

ComfyUI core at our pinned commit (`eb0686bb`) emits
`'function' object has no attribute 'register'` because the node-replacement
API surface is incomplete at that SHA. Bumping `COMFYUI_COMMIT` to a newer
tag should silence it. Pure cosmetic — no functional impact.

### 9. Auto-close drawer when user navigates away from header

Currently relies on document-level click listener. Works but has a
microsecond race when the click target is between elements. Could use
`pointerleave` on the drawer instead.

## Cost-of-running

### 10. Trim ZeroGPU duration cap

Currently `@spaces.GPU(duration=300)` reserves 5 min per call. For Fast preset
(distilled 8 steps) actual usage is ~30 s. Could shorten to 120 s — improves
queue priority for the user (per HF docs). Use dynamic duration based on
preset.

### 11. Local-perf "low-VRAM" path for style mode (GGUF Q4 transformer)

Style mode on Apple Silicon runs ~37× slower per sampling step than the other
modes (~596 s/step on Mac vs ~16 s/step for lipsync). Root cause is
architectural — `LTXAddVideoICLoRAGuide` concatenates the source video's
DWPose latents into the noisy target latent, doubling the attention sequence
to ~56 k tokens. Combined with MPS having no flash-attn-2 and the 22B BF16
model approaching the working-memory ceiling, perf collapses on Mac.

H200 handles this fine (flash-attn-3 + tensor cores + dedicated VRAM ⇒
~30–60 s end to end on Spaces). So this is fundamentally a Mac/MPS gap, not
a code bug.

A "Low VRAM" preset that swaps the BF16 transformer for the GGUF Q4
quantized one would reduce per-step memory pressure and may bring local
style perf into the workable range (still slow, but maybe ~60–90 s/step
instead of 600). The GGUF file is already declared in `MODEL_REGISTRY`
(`UnetLoaderGGUF` consumer). What's missing:

1. A workflow toggle that swaps `UNETLoader` → `UnetLoaderGGUF` for the main
   transformer in style.json (and other modes that benefit).
2. A UI control on the Advanced accordion: "Low VRAM (GGUF Q4)".
3. Wire-through in `_style_parameterize` (and friends) to flip the loader
   class.
4. Delete the matching BF16 path nodes when GGUF is selected (or set them
   to bypass) so we don't load both.

Risk: GGUF transformers behave slightly differently from BF16 — output
quality drops, especially for IC-LoRA paths where the dynamic range matters.
Should be opt-in only, never default. Probably v1.1+ scope (it's listed in
"Out of scope for v1" in CLAUDE.md as the GGUF Q4 / Low VRAM preset).