AGENTS.md

AGENTS.md

Tool-agnostic agent guidance for the ltx2.3-AIO-generator repo. If you're driving Claude Code, Cursor, Aider, Codex, or anything else with file-edit + shell access, start here.

This file is the authoritative project rulebook.

• CLAUDE.md — Claude-specific extensions and the full gotchas catalogue (what & why).
• SKILLS.md — process / how-to (debugging, deployment, when to commit, useful one-liners).
• README.md — public-facing project intro (different audience).

---

TL;DR — the seven rules

1. Mayank Gupta is sole author on every commit. No agent co-author trailers. No "generated with…" footers. No --author= flag. Strip any tool-suggested attribution.
2. Backend is ComfyUI in library mode. We call comfy.execution.PromptExecutor directly with our parameterized workflow JSONs. We do NOT subprocess ComfyUI and we do NOT swap to a different inference engine.
3. Six mode workflows live in workflows/ as user-exported API-format JSON. Do not hand-edit. The user re-exports from the ComfyUI editor when the master changes; python tools/extract_modes.py --master ... --out workflows regenerates the six mode files.
4. Models live in the HF cache. Local: ~/.cache/huggingface/hub symlinked into comfyui/models/<comfy_type>/. Spaces: build-time preload_from_hub + runtime mirror to ~/hf-cache-rw/. Never commit *.safetensors, *.gguf, *.bin, *.pt.
5. Don't pin spaces in requirements.txt. HF Spaces' ZeroGPU build injects its own version; pinning causes pip-resolve failure.
6. HF Space deploys from main. Local default branch is master. Push with git push space master:main — bare git push space master creates an orphan remote branch that does NOT trigger a deploy.
7. VRAM is ComfyUI's job. The only empty_cache() calls live in backend.py's try/finally. Don't sprinkle them elsewhere.

If you can't satisfy any of these without changing architectural shape, ask the user before proceeding.

---

Project shape

Single-process Gradio 5.50 app, flat top-level Python layout, 3.5 k LOC excluding the ComfyUI submodule. ComfyUI itself is vendored as a git submodule locally and runtime-cloned into `/comfyui` on HF Spaces.

app.py            Gradio Blocks entry · _bootstrap · _on_generate · header drawer
backend.py        ComfyUILibraryBackend · @spaces.GPU · _execute_workflow · duration estimator
modes.py          MODE_REGISTRY + per-mode parameterize_fn + node-id constants
models.py         MODEL_REGISTRY · walk_workflow_for_models · ensure_models_for_mode
ui.py             render_status · _render_idle · mode-form layout primitives
workflow.py       load_template · set_input helpers
workflows/        Six API-format mode JSONs — DO NOT hand-edit
assets/           Seed input placeholders for cold-start staging
tools/            extract_modes.py — regenerate workflows/ from a master export
docs/superpowers/ Spec + plan + brainstorm artifacts (per feature)
tests/            L1 + L2 + L3; GPU smoke gated by --gpu
comfyui/          Submodule (local) / runtime clone target (Spaces)

Same code path everywhere. The only branching is in _bootstrap() (cache-mirror dance on Spaces; plain symlink locally) and the @spaces.GPU decorator (identity off Spaces).

---

Locked architecture decisions

These came out of 100+ commits of iteration. Do not relitigate.

Decision	Why	Code reference
ComfyUI library mode (no subprocess)	Direct executor access; shared Python process for model lifecycle and progress reporting	backend.ComfyUILibraryBackend.__init__
Six API-format workflow JSONs	API format ({node_id: {class_type, inputs}}) is what PromptExecutor + walk_workflow_for_models consume. Editor format silently fails.	workflows/*.json
Workflow parameterization via patches	The user owns workflow shape via ComfyUI editor exports; we only patch leaf inputs. Never hand-edit JSON.	modes.parameterize_fn
Custom nodes pinned to SHAs (not branches)	Reproducible builds; git clone --branch <SHA> is unsupported — we use a _git_clone() init+fetch+checkout helper	app.CUSTOM_NODES_PINNED, app._git_clone
HF cache → comfyui/models/<type>/ symlinks	Avoids duplicate weight copies; HF cache stays the single source of truth	models.symlink_hf_cache_to_comfy_layout
_mirror_preload_hf_cache() on Spaces	preload_from_hub files are owned by the build user (root-ish); the runtime user (uid 1000) can't write to them. Hardlink blobs + copy refs into a writable mirror under ~/hf-cache-rw/.	app._mirror_preload_hf_cache
_comfy_dir() per-platform	~/comfyui on Spaces (writable HOME); <repo>/comfyui locally. Both app.py and backend.py must agree.	app._comfy_dir, backend._comfy_dir
@spaces.GPU(duration=callable) applied module-level	Runtime decoration isn't detected by ZeroGPU's startup analyzer. Static decorator + dynamic-duration callable is the supported pattern.	backend._execute_workflow
Per-call duration estimator	A one-size-fits-all 600 s caps everything in the slow queue lane. Estimator reads frames + mode + preset + cold-cache buffer, clamps [60, 900] s.	backend._duration_for
Auto-retry once at 2× on timeout	"GPU task aborted" is the queue-eviction signal; one retry catches transient busy queues.	app._on_generate
allowed_paths=[output_dir] on launch	Gradio 5 refuses files outside cwd / temp / allowed_paths. ComfyUI writes to ~/comfyui/output/... on Spaces — outside the app cwd.	app.app.launch(...)
Header z-index: 15 default / 60 elevated	HF injects #huggingface-space-header at fixed z-index 20. Default keeps our header below it (HF widget visible); drawer-open bumps above the scrim.	_CUSTOM_CSS .aio-header
Click-outside dismisser in gr.Blocks(head=…)	Gradio strips <script> tags inside gr.HTML. Inline scripts have to live in <head> to actually run.	app._HEAD_HTML
Mode tag in header via inline onclick	onclick="…" attributes on plain HTML buttons survive sanitization (unlike inline <script>). Lets us update the tag without a server round-trip.	mode buttons in build_app
Topaz Cinema Slate theme	Locked from brainstorming round. Slate #1A1F26 + amber accent #E0A458 + IBM Plex Sans.	app._TOPAZ_THEME, _CUSTOM_CSS

---

Commit rules

• Conventional Commits: <type>(<scope>): <subject>
  • types: feat, fix, chore, docs, test, refactor, ci, perf
• Subject is imperative, lowercase, no trailing period.
• Body explains why when not obvious. Reference plan task IDs when implementing a specific plan step.
• Frequent small commits; one logical change per commit.
• No agent attribution in commit message or body. See rule 1.
• Don't git push --force to master / main unless the user explicitly says so.

---

Verification rules

• Tests must pass before committing. python -m pytest tests/ -q from the project root. Default skips GPU markers.
• Ruff must be clean. ruff check . && ruff format --check . — both no-op.
• Smoke import after backend / app edits. python -c "import app; b = app.build_app(); print(type(b).__name__)" should print Blocks without traceback. Catches most syntax / import-cycle issues without spinning up the full server.
• For UI changes: start the local server (python app.py → http://127.0.0.1:7860), click through the affected mode, verify visually. Don't trust a green test run + clean ruff as proof the UI works — the test suite doesn't render Gradio Blocks.
• For deployment changes: push to HF Space, watch the build stage transitions (BUILDING → APP_STARTING → RUNNING), verify the runtime stage hits RUNNING before claiming success.

If a change requires breaking these rules, write the reason in the commit body.

---

Testing conventions

• TDD per the plan. Failing test first, then implementation.
• L1 — unit tests on pure Python (mode registry, parameterize_fn, walker, extractor). Runs in CI without GPU.
• L2 — graph validation against the bundled ComfyUI (pytest --comfy-real). Optional; runs locally + nightly.
• L3 — backend smoke tests with real workflow JSONs but stubbed HTTP / filesystem boundaries. Runs in CI without GPU.
• L4 — HF Space smoke. Manual click-through on the live Space after each deploy.
• No mocks for ComfyUI core. Tests run against real workflow JSONs. Stub only HTTP boundaries (HF Hub) and filesystem (use tmp_path and the fake_hf_cache fixture).
• pyproject.toml declares the gpu marker; pass --gpu to opt into GPU smoke.

---

Out of scope (v1 — don't add without asking)

The spec at docs/superpowers/specs/2026-04-30-ltx23-aio-generator-design.md § 11 calls these out as v1.1+. If you find yourself "while I'm here"-ing into one of them, stop.

• Lite mode (LTX23_AIO_LITE=1) for the free HF Spaces tier
• Custom LoRA add/remove rows (Power-Lora-Loader clone)
• GGUF Q4 transformer / "Low VRAM" preset (currently always BF16-served)
• Auto-launch the user's external ComfyUI (LTX23_AIO_COMFYUI_URL)
• Multi-prompt queueing
• Output history persistence across sessions
• Visual regression tests for the Gradio UI
• Property-based / fuzz testing of workflow parameters
• Persistent Storage add-on integration (see docs/future_improvements.md item 6)
• Telemetry-driven duration estimator (requires persistent storage)

If a feature you're adding requires one of these as a sub-step, ask the user.

---

When you're not sure

1. Read docs/superpowers/specs/2026-04-30-ltx23-aio-generator-design.md — that's the architectural source of truth.
2. Read docs/superpowers/plans/2026-04-30-ltx23-aio-generator.md — task-by-task breakdown.
3. Read SKILLS.md — process rules, debugging patterns, deployment workflow, useful one-liners.
4. Read CLAUDE.md — gotchas catalogue and what-not-to-do.
5. git log --oneline — every non-obvious decision has a fix-commit explaining the reasoning.
6. Ask the user. A clarifying question costs the user ten seconds. A wrong implementation costs everyone an hour.