je-suis-tm/marvel_zombies_style_lora_flux2_nf4

Marvel Zombies Style Lora Flux2 NF4

Prompt

Training With QLoRA: Marvel Zombies Style, this image portrays a decaying gas station, a stark monument to abandonment. Peeling paint, cracked concrete, and crumbling interiors define the ruin, with overgrown vegetation reclaiming the space. Two figures stand within: one stand next to a bike, the other leaning against a wall. A desolate field stretches into the distance, punctuated by trees and power lines, amplifying the atmosphere of decay and isolation.

Prompt

Training With QLoRA: Marvel Zombies Style, A lone motorcyclist with a passenger, silhouetted, speeds into a fiery sunset across a barren landscape. The sky bleeds orange and blue, punctuated by scattered clouds. Distant power and telephone poles punctuate the scene, hinting at a decaying world. The rider's form is stark against the vibrant hues, suggesting a thrilling escape. A sense of freedom and adventure permeates the image, tinged with a melancholic beauty.

Prompt

Training With QLoRA: Marvel Zombies Style, this image portrays a colossal cave of towering blue stone blocks, dramatically lit with vibrant, ethereal light. Irregularly stacked stones create immense scale, while orange lights on platforms punctuate the cool blue tones. A large archway dominates the background, contributing to a striking composition. numerous figures stand at the bottom left with the spaceship parked at the bottom right of the image. The style evokes fantasy, inspiring wonder and awe within a cavernous, otherworldly space.

Prompt

Training With QLoRA: Marvel Zombies Style, A warrior stands poised in the foreground, arms extended, wielding two upward angled swords. She stands atop a blue surface stained with blood drops. The vibrant background explodes with green and yellow hues, a darker green dominating the left. Bright highlights suggest intense light sources above the figure, creating a dynamic and potentially chaotic scene.

Prompt

Training With QLoRA: Marvel Zombies Style, the image portrays a vast, brightly lit room with a towering ceiling, likely a storage area. A sleek, modern motorcycle dominates the foreground, centered on a concrete floor. two rows of SUVs and motorcycles in the background. Harsh fluorescent lights lining the walls create an unsettlingly bright and sterile atmosphere. The scene suggests a functional space for vehicle upkeep with the intense lighting and stark environment.

Prompt

Training With QLoRA: Marvel Zombies Style, A powerful figure dominates a dramatic procession in this cinematic fantasy scene. Clad in a black robe with gold trim, glowing red eyes and a striking white hairpiece, he command attention, armed with long spear. A fiery glow from behind casts an ominous light, highlighting their authority. Numerous dark figures forming an army with glowing red eyes stand in the background. The dark atmosphere is thick with tension and foreboding, creating a sense of impending drama as they lead the procession.

Prompt

Training Without QLoRA: Marvel Zombies Style, this image portrays a decaying gas station, a stark monument to abandonment. Peeling paint, cracked concrete, and crumbling interiors define the ruin, with overgrown vegetation reclaiming the space. Two figures stand within: one stand next to a bike, the other leaning against a wall. A desolate field stretches into the distance, punctuated by trees and power lines, amplifying the atmosphere of decay and isolation.

Prompt

Training Without QLoRA: Marvel Zombies Style, A lone motorcyclist with a passenger, silhouetted, speeds into a fiery sunset across a barren landscape. The sky bleeds orange and blue, punctuated by scattered clouds. Distant power and telephone poles punctuate the scene, hinting at a decaying world. The rider's form is stark against the vibrant hues, suggesting a thrilling escape. A sense of freedom and adventure permeates the image, tinged with a melancholic beauty.

Prompt

Training Without QLoRA: Marvel Zombies Style, this image portrays a colossal cave of towering blue stone blocks, dramatically lit with vibrant, ethereal light. Irregularly stacked stones create immense scale, while orange lights on platforms punctuate the cool blue tones. A large archway dominates the background, contributing to a striking composition. numerous figures stand at the bottom left with the spaceship parked at the bottom right of the image. The style evokes fantasy, inspiring wonder and awe within a cavernous, otherworldly space.

Prompt

Training Without QLoRA: Marvel Zombies Style, A warrior stands poised in the foreground, arms extended, wielding two upward angled swords. She stands atop a blue surface stained with blood drops. The vibrant background explodes with green and yellow hues, a darker green dominating the left. Bright highlights suggest intense light sources above the figure, creating a dynamic and potentially chaotic scene.

Prompt

Training Without QLoRA: Marvel Zombies Style, the image portrays a vast, brightly lit room with a towering ceiling, likely a storage area. A sleek, modern motorcycle dominates the foreground, centered on a concrete floor. two rows of SUVs and motorcycles in the background. Harsh fluorescent lights lining the walls create an unsettlingly bright and sterile atmosphere. The scene suggests a functional space for vehicle upkeep with the intense lighting and stark environment.

Prompt

Training Without QLoRA: Marvel Zombies Style, A powerful figure dominates a dramatic procession in this cinematic fantasy scene. Clad in a black robe with gold trim, glowing red eyes and a striking white hairpiece, he command attention, armed with long spear. A fiery glow from behind casts an ominous light, highlighting their authority. Numerous dark figures forming an army with glowing red eyes stand in the background. The dark atmosphere is thick with tension and foreboding, creating a sense of impending drama as they lead the procession.

Prompt

Testing With QLoRA: Marvel Zombies Style, A terrified female archaeologist running through a dense jungle at night, chased by an army of hundreds of undead zombies emerging from the shadows and fog. The scene is lit only by her headlamp and faint moonlight filtering through the thick jungle canopy. The jungle floor is wet and muddy, with twisted roots, broken branches, and ancient stone ruins partially hidden by overgrowth. Her face shows panic and exhaustion as she glances back at the horde.

Prompt

Testing With QLoRA: Marvel Zombies Style, an Aztec god with a skull face, depicted in a full-body view. He stands firmly, gazing intensely and pointing directly at the viewer, conveying a powerful sense of anger. The god wears an intricately designed headdress made of vibrant feathers and jewels, featuring ancient symbols. His elaborate armor, decorated with traditional Aztec motifs, covers his entire body, showcasing intricate details. His eyes glow a menacing red, enhancing his intimidating presence. The background is a deep, shadowy darkness, emphasizing the eerie atmosphere, with subtle glimpses of ancient ruins barely visible in the dim light. The textures of his skin and the shine of the jewels contribute an extra layer of realism, while the full-body composition emphasizes his power and stature

Prompt

Testing With QLoRA: Marvel Zombies Style, Gothic Fire-Breathing winged Dragon in a stone courtyard being Roaring into the night sky

Prompt

Testing Without QLoRA: Marvel Zombies Style, A terrified female archaeologist running through a dense jungle at night, chased by an army of hundreds of undead zombies emerging from the shadows and fog. The scene is lit only by her headlamp and faint moonlight filtering through the thick jungle canopy. The jungle floor is wet and muddy, with twisted roots, broken branches, and ancient stone ruins partially hidden by overgrowth. Her face shows panic and exhaustion as she glances back at the horde.

Prompt

Testing Without QLoRA: Marvel Zombies Style, an Aztec god with a skull face, depicted in a full-body view. He stands firmly, gazing intensely and pointing directly at the viewer, conveying a powerful sense of anger. The god wears an intricately designed headdress made of vibrant feathers and jewels, featuring ancient symbols. His elaborate armor, decorated with traditional Aztec motifs, covers his entire body, showcasing intricate details. His eyes glow a menacing red, enhancing his intimidating presence. The background is a deep, shadowy darkness, emphasizing the eerie atmosphere, with subtle glimpses of ancient ruins barely visible in the dim light. The textures of his skin and the shine of the jewels contribute an extra layer of realism, while the full-body composition emphasizes his power and stature

Prompt

Testing Without QLoRA: Marvel Zombies Style, Gothic Fire-Breathing winged Dragon in a stone courtyard being Roaring into the night sky

All files are also archived in https://github.com/je-suis-tm/huggingface-archive in case this gets censored.

Marvel Zombies is Marvel animation series in 2025. This QLoRA intends to replicate that animation style.

The training is based on https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/test_dreambooth_lora_flux2.py. Everything in this training script needs to be set at the same torch dtype as the script was designed for unquantized version which will export in float32. The training took 11.5 hours on A100 80GB with max VRAM consumption at 32GB. The inference consumes 35GB of VRAM. To avoid running low on VRAM, both transformers and text_encoder were quantized.

Train

export MODEL_NAME="diffusers/FLUX.2-dev-bnb-4bit"
export INSTANCE_DIR="/pvol/marvel_zombies_style"
export OUTPUT_DIR="/pvol/marvel_zombies_style_lora_flux2_nf4"
export Q_DIR="/pvol/quantization_config.json"
accelerate config default
accelerate launch train_dreambooth_lora_flux2.py \
  --pretrained_model_name_or_path=$MODEL_NAME  \
  --mixed_precision="bf16" \
  --dataset_name=$INSTANCE_DIR \
  --output_dir=$OUTPUT_DIR \
  --bnb_quantization_config_path=$Q_DIR \
  --gradient_checkpointing \
  --cache_latents \
  --instance_prompt="Marvel Zombies style" \
  --caption_column="text" \
  --aspect_ratio_buckets="720,1728" \
  --center_crop \
  --train_batch_size=1 \
  --guidance_scale=1 \
  --use_8bit_adam \
  --offload \
  --checkpointing_steps=100 \
  --gradient_accumulation_steps=4 \
  --optimizer="adamW" \
  --learning_rate=1e-4 \
  --lr_scheduler="constant" \
  --lr_warmup_steps=100 \
  --max_train_steps=1500 \
  --rank=4 \
  --seed="0" 

Usage

import torch
from transformers import Mistral3ForConditionalGeneration

from diffusers import Flux2Pipeline, Flux2Transformer2DModel

repo_id = "diffusers/FLUX.2-dev-bnb-4bit"
device = "cuda:0"
torch_dtype = torch.float32 #only supports float32 when using train_dreambooth_lora_flux2.py 

transformer = Flux2Transformer2DModel.from_pretrained(
  repo_id, subfolder="transformer", torch_dtype=torch_dtype, device_map="cuda:0"
)
text_encoder = Mistral3ForConditionalGeneration.from_pretrained(
  repo_id, subfolder="text_encoder", dtype=torch_dtype, device_map="cuda:0"
)

pipe = Flux2Pipeline.from_pretrained(
  repo_id, transformer=transformer, text_encoder=text_encoder, torch_dtype=torch_dtype
)
pipe.load_lora_weights("je-suis-tm/marvel_zombies_style_lora_flux2_nf4",
                       weight_name='pytorch_lora_weights.safetensors')
pipe.enable_model_cpu_offload()

prompt = "Marvel Zombies Style, an Aztec god with a skull face, depicted in a full-body view. He stands firmly, gazing intensely and pointing directly at the viewer, conveying a powerful sense of anger. The god wears an intricately designed headdress made of vibrant feathers and jewels, featuring ancient symbols. His elaborate armor, decorated with traditional Aztec motifs, covers his entire body, showcasing intricate details. His eyes glow a menacing red, enhancing his intimidating presence. The background is a deep, shadowy darkness, emphasizing the eerie atmosphere, with subtle glimpses of ancient ruins barely visible in the dim light. The textures of his skin and the shine of the jewels contribute an extra layer of realism, while the full-body composition emphasizes his power and stature"

image = pipe(
  prompt=prompt,
  generator=torch.Generator(device=device).manual_seed(42),
  num_inference_steps=50, # 28 is a good trade-off
  guidance_scale=4,
  height=800, 
  width=1920, #the movie was in a weird resolution 800*1920, the resolution needs to be divided by 16
).images[0]

image.save("marvel_zombies_style.png")

Trigger words

You should use Marvel Zombies Style to trigger the image generation.

Download model

Download them in the Files & versions tab.