joyfox
/

Wan2.1-Fight-LoRA

+---
+license: apache-2.0
+language:
+- en
+base_model:
+- Wan-AI/Wan2.1-I2V-14B-480P
+- Wan-AI/Wan2.1-I2V-14B-480P-Diffusers
+pipeline_tag: image-to-video
+tags:
+- text-to-image
+- lora
+- diffusers
+- template:diffusion-lora
+widget:
+- text: >-
+    In the video, a miniature dog is presented. The dog is held in a person's
+    hands. The person then presses on the dog, causing a sq41sh squish effect.
+    The person keeps pressing down on the dog, further showing the sq41sh squish
+    effect.
+  output:
+    url: example_videos/dog_squish.mp4
+- text: >-
+    In the video, a miniature tank is presented. The tank is held in a person's
+    hands. The person then presses on the tank, causing a sq41sh squish effect.
+    The person keeps pressing down on the tank, further showing the sq41sh
+    squish effect.
+  output:
+    url: example_videos/tank_squish.mp4
+- text: >-
+    In the video, a miniature balloon is presented. The balloon is held in a
+    person's hands. The person then presses on the balloon, causing a sq41sh
+    squish effect. The person keeps pressing down on the balloon, further
+    showing the sq41sh squish effect.
+  output:
+    url: example_videos/balloon_squish.mp4
+- text: >-
+    In the video, a miniature rodent is presented. The rodent is held in a
+    person's hands. The person then presses on the rodent, causing a sq41sh
+    squish effect. The person keeps pressing down on the rodent, further showing
+    the sq41sh squish effect.
+  output:
+    url: example_videos/rodent_squish.mp4
+- text: >-
+    In the video, a miniature person is presented. The person is held in a
+    person's hands. The person then presses on the person, causing a sq41sh
+    squish effect. The person keeps pressing down on the person, further showing
+    the sq41sh squish effect.
+  output:
+    url: example_videos/person_squish.mp4
+---
+<div style="background-color: #f8f9fa; padding: 20px; border-radius: 10px; margin-bottom: 20px;">
+  <h1 style="color: #24292e; margin-top: 0;">Squish Effect LoRA for Wan2.1 14B I2V 480p</h1>
+  <div style="background-color: white; padding: 15px; border-radius: 8px; margin: 15px 0; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
+    <h2 style="color: #24292e; margin-top: 0;">Overview</h2>
+    <p>This LoRA is trained on the Wan2.1 14B I2V 480p model and allows you to squish any object in an image. The effect works on a wide variety of objects, from animals to vehicles to people!</p>
+  </div>
+  <div style="background-color: white; padding: 15px; border-radius: 8px; margin: 15px 0; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
+    <h2 style="color: #24292e; margin-top: 0;">Features</h2>
+    <ul style="margin-bottom: 0;">
+      <li>Transform any image into a video of it being squished</li>
+      <li>Trained on the Wan2.1 14B 480p I2V base model</li>
+      <li>Consistent results across different object types</li>
+      <li>Simple prompt structure that's easy to adapt</li>
+    </ul>
+  </div>
+  <div style="background-color: white; padding: 15px; border-radius: 8px; margin: 15px 0; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
+  <h2 style="color: #24292e; margin-top: 0;">Community</h2>
+  <ul style="margin-bottom: 0;">
+    <li>
+      Generate videos with 100+ Camera Control and VFX LoRAs on the
+      <a href="https://app.remade.ai/canvas/create" style="color: #0366d6; text-decoration: none;">Remade Canvas</a>.
+    </li>
+    <li>
+      <b>Discord:</b>
+      <a href="https://remade.ai/join-discord?utm_source=Huggingface&utm_medium=Social&utm_campaign=model_release&utm_content=crash_zoom_out" style="color: #0366d6; text-decoration: none;">
+        Join our community
+      </a> to generate videos with this LoRA for free
+    </li>
+  </ul>
+</div>
+<Gallery />
+# Model File and Inference Workflow
+## 📥 Download Links:
+- [squish_18.safetensors](./squish_18.safetensors) - LoRA Model File
+- [wan_img2video_lora_workflow.json](./workflow/wan_img2video_lora_workflow.json) - Wan I2V with LoRA Workflow for ComfyUI
+## Using with Diffusers
+```py
+pip install git+https://github.com/huggingface/diffusers.git
+```
+```py
+import torch
+from diffusers.utils import export_to_video, load_image
+from diffusers import AutoencoderKLWan, WanImageToVideoPipeline
+from transformers import CLIPVisionModel
+import numpy as np
+model_id = "Wan-AI/Wan2.1-I2V-14B-480P-Diffusers"
+image_encoder = CLIPVisionModel.from_pretrained(model_id, subfolder="image_encoder", torch_dtype=torch.float32)
+vae = AutoencoderKLWan.from_pretrained(model_id, subfolder="vae", torch_dtype=torch.float32)
+pipe = WanImageToVideoPipeline.from_pretrained(model_id, vae=vae, image_encoder=image_encoder, torch_dtype=torch.bfloat16)
+pipe.to("cuda")
+pipe.load_lora_weights("Remade/Squish")
+pipe.enable_model_cpu_offload() #for low-vram environments
+prompt = "In the video, a miniature cat toy is presented. The cat toy is held in a person's hands. The person then presses on the cat toy, causing a sq41sh squish effect. The person keeps pressing down on the cat toy, further showing the sq41sh squish effect."
+image = load_image("https://huggingface.co/datasets/diffusers/cat_toy_example/resolve/main/1.jpeg")
+max_area = 480 * 832
+aspect_ratio = image.height / image.width
+mod_value = pipe.vae_scale_factor_spatial * pipe.transformer.config.patch_size[1]
+height = round(np.sqrt(max_area * aspect_ratio)) // mod_value * mod_value
+width = round(np.sqrt(max_area / aspect_ratio)) // mod_value * mod_value
+image = image.resize((width, height))
+output = pipe(
+    image=image,
+    prompt=prompt,
+    height=height,
+    width=width,
+    num_frames=81,
+    guidance_scale=5.0,
+    num_inference_steps=28
+).frames[0]
+export_to_video(output, "output.mp4", fps=16)
+```
+---
+<div style="background-color: #f8f9fa; padding: 20px; border-radius: 10px; margin-bottom: 20px;">
+  <div style="background-color: white; padding: 15px; border-radius: 8px; margin: 15px 0; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
+    <h2 style="color: #24292e; margin-top: 0;">Recommended Settings</h2>
+    <ul style="margin-bottom: 0;">
+      <li><b>LoRA Strength:</b> 1.0</li>
+      <li><b>Embedded Guidance Scale:</b> 6.0</li>
+      <li><b>Flow Shift:</b> 5.0</li>
+    </ul>
+  </div>
+  <div style="background-color: white; padding: 15px; border-radius: 8px; margin: 15px 0; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
+    <h2 style="color: #24292e; margin-top: 0;">Trigger Words</h2>
+    <p>The key trigger phrase is: <code style="background-color: #f0f0f0; padding: 3px 6px; border-radius: 4px;">sq41sh squish effect</code></p>
+  </div>
+  <div style="background-color: white; padding: 15px; border-radius: 8px; margin: 15px 0; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
+    <h2 style="color: #24292e; margin-top: 0;">Prompt Template</h2>
+    <p>For best results, use this prompt structure:</p>
+    <div style="background-color: #f0f0f0; padding: 12px; border-radius: 6px; margin: 10px 0;">
+      <i>In the video, a miniature [object] is presented. The [object] is held in a person's hands. The person then presses on the [object], causing a sq41sh squish effect. The person keeps pressing down on the [object], further showing the sq41sh squish effect.</i>
+    </div>
+    <p>Simply replace <code style="background-color: #f0f0f0; padding: 3px 6px; border-radius: 4px;">[object]</code> with whatever you want to see squished!</p>
+  </div>
+  <div style="background-color: white; padding: 15px; border-radius: 8px; margin: 15px 0; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
+    <h2 style="color: #24292e; margin-top: 0;">ComfyUI Workflow</h2>
+    <p>This LoRA works with a modified version of <a href="https://github.com/kijai/ComfyUI-WanVideoWrapper/blob/main/example_workflows/wanvideo_480p_I2V_example_02.json" style="color: #0366d6; text-decoration: none;">Kijai's Wan Video Wrapper workflow</a>. The main modification is adding a Wan LoRA node connected to the base model.</p>
+    <img src="./workflow/workflow_screenshot.png" style="width: 100%; border-radius: 8px; margin: 15px 0; box-shadow: 0 4px 8px rgba(0,0,0,0.1);">
+    <p>See the Downloads section above for the modified workflow.</p>
+  </div>
+</div>
+<div style="background-color: #f8f9fa; padding: 20px; border-radius: 10px; margin-bottom: 20px;">
+  <div style="background-color: white; padding: 15px; border-radius: 8px; margin: 15px 0; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
+    <h2 style="color: #24292e; margin-top: 0;">Model Information</h2>
+    <p>The model weights are available in Safetensors format. See the Downloads section above.</p>
+  </div>
+  <div style="background-color: white; padding: 15px; border-radius: 8px; margin: 15px 0; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
+    <h2 style="color: #24292e; margin-top: 0;">Training Details</h2>
+    <ul style="margin-bottom: 0;">
+      <li><b>Base Model:</b> Wan2.1 14B I2V 480p</li>
+      <li><b>Training Data:</b> 1.5 minutes of video (20 short clips of things being squished)</li>
+      <li><b>Epochs:</b> 18</li>
+    </ul>
+  </div>
+  <div style="background-color: white; padding: 15px; border-radius: 8px; margin: 15px 0; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
+    <h2 style="color: #24292e; margin-top: 0;">Additional Information</h2>
+    <p>Training was done using <a href="https://github.com/tdrussell/diffusion-pipe" style="color: #0366d6; text-decoration: none;">Diffusion Pipe for Training</a></p>
+  </div>
+  <div style="background-color: white; padding: 15px; border-radius: 8px; margin: 15px 0; box-shadow: 0 2px 4px rgba(0,0,0,0.1);">
+    <h2 style="color: #24292e; margin-top: 0;">Acknowledgments</h2>
+    <p style="margin-bottom: 0;">Special thanks to Kijai for the ComfyUI Wan Video Wrapper and tdrussell for the training scripts!</p>
+  </div>
+</div>