je-suis-tm/wong_kar_wai_fallen_angels_lora_flux2_nf4

Wong Kar Wai Fallen Angels style Lora Flux2 NF4

Prompt

Training With QLoRA: WKW Style, A dimly lit tunnel, concrete walls echoing with dust, leads to a vast, empty stadium. A single fluorescent light casts an eerie glow on the floor, stretching a long shadow. Two figures in football uniforms stand embraced each other in darkness at the tunnel's entrance, their faces unseen. Beyond, a green field stretches into the distance, stadium lights shining warmly. The scene evokes anticipation and foreboding.

Prompt

Training With QLoRA: WKW Style, A tilted view of vibrant Hong Kong street pulses with nighttime energy. Neon signs blaze against a dark sky, reflecting in the wet pavement, evidence of recent rain or bustling traffic. Businesses glow warmly, their signs a riot of color. Streetlights cast a golden hue, amplifying the lively atmosphere. The scene crackles with excitement and entertainment, a sensory overload of light and activity, capturing the raw, electric pulse of urban life.

Prompt

Training With QLoRA: WKW Style, A hand, fingers curled in a delicate, holds a lit cigarette. The glow illuminates her hand against a shadowy backdrop, smoke curling upwards like a silent question. The darkness amplifies the cigarette's brightness, creating depth and a sense of quiet contemplation.

Prompt

Training With QLoRA: WKW Style, a woman with long, dark hair leans against a jukebox emitting yellow light, her black top accented with color and small red dot necklace. Positioned right of frame, she tilts her head, intently studying the ornate jukebox on the left, framed by a vibrant yellow orange glow. Her face is obscured by her hair and bang. The background fades into darkness, creating a dramatic contrast.

Prompt

Training With QLoRA: WKW Style, black and white picture with grain effects shows a man leans against a stark, white tiled wall, head tilted back, arms crossed on his chest. Dressed in dark suit, he's angled slightly to the camera, creating depth. The minimalist background, devoid of other elements, isolates him completely. The image evokes a somber, contemplative mood, suggesting a man lost in thought, emphasized by the dark tones and uncluttered scene.

Prompt

Training With QLoRA: WKW Style, a cinematic still depicts an Asian woman absorbed in a magazine within a softly lit washing room. She sits poised on a white washing machine, a red shawl adding warmth to her posture. Her black and white patterned dress contrasts with her long, dark hair. The scene, evokes a quiet intimacy. Light green walls complete the mood, suggesting a moment of solitary reflection.

Prompt

Training Without QLoRA: WKW Style, A dimly lit tunnel, concrete walls echoing with dust, leads to a vast, empty stadium. A single fluorescent light casts an eerie glow on the floor, stretching a long shadow. Two figures in football uniforms stand embraced each other in darkness at the tunnel's entrance, their faces unseen. Beyond, a green field stretches into the distance, stadium lights shining warmly. The scene evokes anticipation and foreboding.

Prompt

Training Without QLoRA: WKW Style, A tilted view of vibrant Hong Kong street pulses with nighttime energy. Neon signs blaze against a dark sky, reflecting in the wet pavement, evidence of recent rain or bustling traffic. Businesses glow warmly, their signs a riot of color. Streetlights cast a golden hue, amplifying the lively atmosphere. The scene crackles with excitement and entertainment, a sensory overload of light and activity, capturing the raw, electric pulse of urban life.

Prompt

Training Without QLoRA: WKW Style, A hand, fingers curled in a delicate, holds a lit cigarette. The glow illuminates her hand against a shadowy backdrop, smoke curling upwards like a silent question. The darkness amplifies the cigarette's brightness, creating depth and a sense of quiet contemplation.

Prompt

Training Without QLoRA: WKW Style, a woman with long, dark hair leans against a jukebox emitting yellow light, her black top accented with color and small red dot necklace. Positioned right of frame, she tilts her head, intently studying the ornate jukebox on the left, framed by a vibrant yellow orange glow. Her face is obscured by her hair and bang. The background fades into darkness, creating a dramatic contrast.

Prompt

Training Without QLoRA: WKW Style, black and white picture with grain effects shows a man leans against a stark, white tiled wall, head tilted back, arms crossed on his chest. Dressed in dark suit, he's angled slightly to the camera, creating depth. The minimalist background, devoid of other elements, isolates him completely. The image evokes a somber, contemplative mood, suggesting a man lost in thought, emphasized by the dark tones and uncluttered scene.

Prompt

Training Without QLoRA: WKW Style, a cinematic still depicts an Asian woman absorbed in a magazine within a softly lit washing room. She sits poised on a white washing machine, a red shawl adding warmth to her posture. Her black and white patterned dress contrasts with her long, dark hair. The scene, evokes a quiet intimacy. Light green walls complete the mood, suggesting a moment of solitary reflection.

Prompt

Testing With QLoRA: WKW Style, a stark black and white still. the woman, pressed her face close on the table, a silent drama unfolding. Her dark hair spills against her shoulder. The background fades into shadow, punctuated by distant window light. A palpable intimacy hangs in the air, a shared vulnerability etched in their posture. Somber, contemplative, a moment suspended.

Prompt

Testing With QLoRA: WKW Style, a blurry view of a woman in the closeup, her red curls framing a contemplative gaze off camera, giving sadness vibe. she wears A white shirt with a dark palm leaf detail. The background, a tunnel with red walls and harsh fluorescent light, amplifies the dramatic mood. This image encapsulates a pivotal, character defining moment.

Prompt

Testing With QLoRA: WKW Style, The image shows a man reclining on his back on top of a counter with a cigarette in hand. He's dressed casually in a loose white shirt and khaki pants, his head tilted upwards. Empty sauce bottles line the counter beside him, arranged neatly. The scene conveys a sense of repose, suggesting a moment of rest or a break, perhaps in a relaxed setting like a cafe. His body language is relaxed, hinting at a quiet, contemplative mood.

Prompt

Testing Without QLoRA: WKW Style, a stark black and white still. the woman, pressed her face close on the table, a silent drama unfolding. Her dark hair spills against her shoulder. The background fades into shadow, punctuated by distant window light. A palpable intimacy hangs in the air, a shared vulnerability etched in their posture. Somber, contemplative, a moment suspended.

Prompt

Testing Without QLoRA: WKW Style, a blurry view of a woman in the closeup, her red curls framing a contemplative gaze off camera, giving sadness vibe. she wears A white shirt with a dark palm leaf detail. The background, a tunnel with red walls and harsh fluorescent light, amplifies the dramatic mood. This image encapsulates a pivotal, character defining moment.

Prompt

Testing Without QLoRA: WKW Style, The image shows a man reclining on his back on top of a counter with a cigarette in hand. He's dressed casually in a loose white shirt and khaki pants, his head tilted upwards. Empty sauce bottles line the counter beside him, arranged neatly. The scene conveys a sense of repose, suggesting a moment of rest or a break, perhaps in a relaxed setting like a cafe. His body language is relaxed, hinting at a quiet, contemplative mood.

All files are also archived in https://github.com/je-suis-tm/huggingface-archive in case this gets censored.

Wong Kar Wai is one of my fav directors. This QLoRA intends to replicate that bold neon color tone, extreme wide angle, surreal neo-noir vibe of the movie Fallen Angels (1995). The comparison between flux1 https://huggingface.co/je-suis-tm/wong_kar_wai_fallen_angels_lora_flux and flux2 is very obvious. Flux1 does not support different aspect ratios in training but it does not distort human faces or female bodies in training dataset. Flux2 can capture the ultra wide angle lens, camera angles and lightings, but it really fucks up the faces and bodies and does not disclose anything about the shady operation which is fucking disgusting. It is up to you to decide which lora reflects the aesthetics of Wong Kar Wai. If flux3 behaves like this, I am switching to wan or qwen, fuck that censorship.

The training is based on https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/test_dreambooth_lora_flux2.py. Everything in this training script needs to be set at the same torch dtype as the script was designed for unquantized version which will export in float32. The training took 11 hours on A100 80GB with max VRAM consumption at 32GB. The inference consumes 35GB of VRAM. To avoid running low on VRAM, both transformers and text_encoder were quantized.

Train

export MODEL_NAME="diffusers/FLUX.2-dev-bnb-4bit"
export INSTANCE_DIR="/pvol/wong_kar_wai_fallen_angels"
export OUTPUT_DIR="/pvol/wong_kar_wai_fallen_angels_lora_flux2_nf4"
export Q_DIR="/pvol/quantization_config.json"
accelerate config default
accelerate launch train_dreambooth_lora_flux2.py \
  --pretrained_model_name_or_path=$MODEL_NAME  \
  --mixed_precision="bf16" \
  --dataset_name=$INSTANCE_DIR \
  --output_dir=$OUTPUT_DIR \
  --bnb_quantization_config_path=$Q_DIR \
  --gradient_checkpointing \
  --cache_latents \
  --instance_prompt="WKW style" \
  --caption_column="text" \
  --aspect_ratio_buckets="800,1456" \
  --center_crop \
  --train_batch_size=1 \
  --guidance_scale=1 \
  --use_8bit_adam \
  --offload \
  --checkpointing_steps=100 \
  --gradient_accumulation_steps=4 \
  --optimizer="adamW" \
  --learning_rate=1e-4 \
  --lr_scheduler="constant" \
  --lr_warmup_steps=100 \
  --max_train_steps=1500 \
  --rank=4 \
  --seed="0" 

Usage

import torch
from transformers import Mistral3ForConditionalGeneration

from diffusers import Flux2Pipeline, Flux2Transformer2DModel

repo_id = "diffusers/FLUX.2-dev-bnb-4bit"
device = "cuda:0"
torch_dtype = torch.float32 #only supports float32 when using train_dreambooth_lora_flux2.py 

transformer = Flux2Transformer2DModel.from_pretrained(
  repo_id, subfolder="transformer", torch_dtype=torch_dtype, device_map="cuda:0"
)
text_encoder = Mistral3ForConditionalGeneration.from_pretrained(
  repo_id, subfolder="text_encoder", dtype=torch_dtype, device_map="cuda:0"
)

pipe = Flux2Pipeline.from_pretrained(
  repo_id, transformer=transformer, text_encoder=text_encoder, torch_dtype=torch_dtype
)
pipe.load_lora_weights("je-suis-tm/wong_kar_wai_fallen_angels_lora_flux2_nf4",
                       weight_name='pytorch_lora_weights.safetensors')
pipe.enable_model_cpu_offload()

prompt = "WKW Style, The image shows a man reclining on his back on top of a counter with a cigarette in hand. He's dressed casually in a loose white shirt and khaki pants, his head tilted upwards. Empty sauce bottles line the counter beside him, arranged neatly. The scene conveys a sense of repose, suggesting a moment of rest or a break, perhaps in a relaxed setting like a cafe. His body language is relaxed, hinting at a quiet, contemplative mood"

image = pipe(
  prompt=prompt,
  generator=torch.Generator(device=device).manual_seed(42),
  num_inference_steps=50, # 28 is a good trade-off
  guidance_scale=4,
  height=800, 
  width=1456, #the movie was in a weird resolution 1056*1920, the resolution needs to be divided by 16
).images[0]

image.save("wong_kar_wai_fallen_angels.png")

Trigger words

You should use WKW style to trigger the image generation.

Download model

Download them in the Files & versions tab.