Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -2,7 +2,6 @@
|
|
| 2 |
license: apache-2.0
|
| 3 |
language:
|
| 4 |
- en
|
| 5 |
-
- zh
|
| 6 |
library_name: other
|
| 7 |
tags:
|
| 8 |
- video-generation
|
|
@@ -18,8 +17,17 @@ pipeline_tag: image-to-video
|
|
| 18 |
|
| 19 |
# LTX2.3-ICEdit-Insight
|
| 20 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
Research-oriented model release for task-aware video restoration and editing under the `LTX-2.3` framework.
|
| 22 |
|
|
|
|
|
|
|
| 23 |
This repository contains:
|
| 24 |
|
| 25 |
- `ltx-2.3-edit-insight-dev-fp8.safetensors`
|
|
@@ -38,24 +46,6 @@ These weights are intended to be used with the project's `run_pipeline.py` workf
|
|
| 38 |
|
| 39 |
From a research perspective, the checkpoint is intended to support controlled video restoration and editing under a shared latent diffusion formulation. The paired IC-LoRA adapters specialize the backbone toward structure-preserving super-resolution and watermark-aware content recovery, while the unified checkpoint packaging keeps the full generative stack aligned for repeatable experiments and downstream ablations.
|
| 40 |
|
| 41 |
-
## 中文说明
|
| 42 |
-
|
| 43 |
-
这是当前项目使用的 Hugging Face 模型仓库,包含一个 Insight 一体化基模和两个任务型 IC-LoRA:
|
| 44 |
-
|
| 45 |
-
- 超分增强:`ltx2.3-video-upscale-v2.safetensors`
|
| 46 |
-
- 视频去水印:`ltx2.3-ic-watermarkeRM.safetensors`
|
| 47 |
-
- Insight 基模:`ltx-2.3-edit-insight-dev-fp8.safetensors`
|
| 48 |
-
|
| 49 |
-
整体定位是面向视频超分、去水印和细节恢复的统一编辑框架。项目在 `LTX-2.3` 基础上使用任务感知型 IC-LoRA,并通过参考 latent 条件引导来增强结构恢复与纹理细节控制。
|
| 50 |
-
|
| 51 |
-
其中 `ltx-2.3-edit-insight-dev-fp8.safetensors` 并不是普通的推理底模打包,而是本项目 Insight 分支的统一研究型基模发布。它把扩散 transformer、video VAE、audio VAE、text projection 与 vocoder 组织为单一 all-in-one checkpoint,用来支撑结构保持型视频修复、细节重建与任务定向编辑的可复现实验设置。
|
| 52 |
-
|
| 53 |
-
当前推荐用法:
|
| 54 |
-
|
| 55 |
-
- 使用本项目的 `run_pipeline.py`
|
| 56 |
-
- 默认使用单阶段推理
|
| 57 |
-
- 按任务切换 LoRA,而不是把两个 LoRA 同时叠加
|
| 58 |
-
|
| 59 |
## English Overview
|
| 60 |
|
| 61 |
This package is built for the Insight version of the project's LTX-2.3 editing pipeline. Instead of shipping only task adapters, it also includes the corresponding Insight base checkpoint so the workflow can be reproduced with the exact model assets used by the project.
|
|
@@ -66,6 +56,18 @@ Recommended usage:
|
|
| 66 |
- keep single-stage inference enabled by default
|
| 67 |
- load one task LoRA at a time depending on the editing goal
|
| 68 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 69 |
## Files
|
| 70 |
|
| 71 |
| File | Purpose |
|
|
@@ -76,17 +78,6 @@ Recommended usage:
|
|
| 76 |
| `assets/effects/output_004.webp` | Effect preview |
|
| 77 |
| `assets/effects/output_005.webp` | Effect preview |
|
| 78 |
|
| 79 |
-
## Super-Resolution Showcase
|
| 80 |
-
|
| 81 |
-
The following previews are included directly from the current project assets.
|
| 82 |
-
|
| 83 |
-
<table>
|
| 84 |
-
<tr>
|
| 85 |
-
<td align="center"><img src="./assets/effects/output_004.webp" alt="Super-resolution preview 1" width="600"/></td>
|
| 86 |
-
<td align="center"><img src="./assets/effects/output_005.webp" alt="Super-resolution preview 2" width="600"/></td>
|
| 87 |
-
</tr>
|
| 88 |
-
</table>
|
| 89 |
-
|
| 90 |
## Usage With This Project
|
| 91 |
|
| 92 |
Run all commands from the project root.
|
|
|
|
| 2 |
license: apache-2.0
|
| 3 |
language:
|
| 4 |
- en
|
|
|
|
| 5 |
library_name: other
|
| 6 |
tags:
|
| 7 |
- video-generation
|
|
|
|
| 17 |
|
| 18 |
# LTX2.3-ICEdit-Insight
|
| 19 |
|
| 20 |
+
<table>
|
| 21 |
+
<tr>
|
| 22 |
+
<td align="center"><img src="./assets/effects/output_004.webp" alt="Super-resolution preview 1" width="600"/></td>
|
| 23 |
+
<td align="center"><img src="./assets/effects/output_005.webp" alt="Super-resolution preview 2" width="600"/></td>
|
| 24 |
+
</tr>
|
| 25 |
+
</table>
|
| 26 |
+
|
| 27 |
Research-oriented model release for task-aware video restoration and editing under the `LTX-2.3` framework.
|
| 28 |
|
| 29 |
+
Project links: [GitHub project](https://github.com/Valiant-Cat/LTX2-ICEdit-Insight) | [Valiant Cat on Hugging Face](https://huggingface.co/valiantcat)
|
| 30 |
+
|
| 31 |
This repository contains:
|
| 32 |
|
| 33 |
- `ltx-2.3-edit-insight-dev-fp8.safetensors`
|
|
|
|
| 46 |
|
| 47 |
From a research perspective, the checkpoint is intended to support controlled video restoration and editing under a shared latent diffusion formulation. The paired IC-LoRA adapters specialize the backbone toward structure-preserving super-resolution and watermark-aware content recovery, while the unified checkpoint packaging keeps the full generative stack aligned for repeatable experiments and downstream ablations.
|
| 48 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 49 |
## English Overview
|
| 50 |
|
| 51 |
This package is built for the Insight version of the project's LTX-2.3 editing pipeline. Instead of shipping only task adapters, it also includes the corresponding Insight base checkpoint so the workflow can be reproduced with the exact model assets used by the project.
|
|
|
|
| 56 |
- keep single-stage inference enabled by default
|
| 57 |
- load one task LoRA at a time depending on the editing goal
|
| 58 |
|
| 59 |
+
## 🧠 Training
|
| 60 |
+
|
| 61 |
+
This model was trained and optimized by the AI Laboratory of Chongqing Valiant Cat Technology Co., LTD.
|
| 62 |
+
|
| 63 |
+
Visit [vvicat.com](https://vvicat.com/) for business collaborations or research partnerships.
|
| 64 |
+
|
| 65 |
+
## 🧩 Integration with ComfyUI
|
| 66 |
+
|
| 67 |
+
This model works with the modified ComfyUI [workflows](https://github.com/Valiant-Cat/LTX2-ICEdit-Insight/tree/main/workflows) provided by the project.
|
| 68 |
+
|
| 69 |
+
For ComfyUI-based editing, load the base model in the UNet-side model path required by the workflow, then attach the task-specific IC-LoRA for the corresponding edit objective.
|
| 70 |
+
|
| 71 |
## Files
|
| 72 |
|
| 73 |
| File | Purpose |
|
|
|
|
| 78 |
| `assets/effects/output_004.webp` | Effect preview |
|
| 79 |
| `assets/effects/output_005.webp` | Effect preview |
|
| 80 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 81 |
## Usage With This Project
|
| 82 |
|
| 83 |
Run all commands from the project root.
|