je-suis-tm/zoe_kravitz_lora_flux

Zoe Kravitz Lora Flux

Prompt

Training With QLoRA: This close-up photograph captures Zoe Kravitz in a striking portrait. The image focuses on her face and upper body, cropped to showcase her head and shoulders. She's turned slightly, revealing her right shoulder and a black dress detailed with feathers or sequins on the straps. Her short buzz cut is styled upwards, framing her face. The background hints at a red carpet event or press conference, featuring a gray wall with large yellow letters partially hidden by her head. The composition is clean and elegant, highlighting her poised demeanor and sophisticated style. Soft, natural lighting enhances her beauty without harsh contrasts, creating a polished and refined portrait of the actress.

Prompt

Training Without QLoRA: This close-up photograph captures Zoe Kravitz in a striking portrait. The image focuses on her face and upper body, cropped to showcase her head and shoulders. She's turned slightly, revealing her right shoulder and a black dress detailed with feathers or sequins on the straps. Her short buzz cut is styled upwards, framing her face. The background hints at a red carpet event or press conference, featuring a gray wall with large yellow letters partially hidden by her head. The composition is clean and elegant, highlighting her poised demeanor and sophisticated style. Soft, natural lighting enhances her beauty without harsh contrasts, creating a polished and refined portrait of the actress.

Prompt

Testing With QLoRA: a synthwave girl named Zoe Kravitz in foreground, buildings along miami beach with neon signs at night in background, retrowave synthwave aesthetics

Prompt

Testing Without QLoRA: a synthwave girl named Zoe Kravitz in foreground, buildings along miami beach with neon signs at night in background, retrowave synthwave aesthetics

All files are also archived in https://github.com/je-suis-tm/huggingface-archive in case this gets censored.

This a non-quantized version of https://huggingface.co/je-suis-tm/zoe_kravitz_lora_flux_nf4. Both are trained on the same dataset. The training is based on https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/test_dreambooth_lora_flux.py. The training took 3 hours on A100 80GB with max VRAM consumption at 35GB. The inference consumes 36GB of VRAM.

Train

    export MODEL_NAME="black-forest-labs/FLUX.1-dev"
    export INSTANCE_DIR="/pvol/zoe_kravitz"
    export OUTPUT_DIR="/pvol/zoe_kravitz_lora_flux"
    accelerate config default
    accelerate launch train_dreambooth_lora_flux1.py \
      --pretrained_model_name_or_path=$MODEL_NAME  \
      --mixed_precision="bf16" \
      --dataset_name=$INSTANCE_DIR \
      --output_dir=$OUTPUT_DIR \
      --gradient_checkpointing \
      --instance_prompt="Zoe Kravitz" \
      --caption_column="text" \
      --resolution=1024 \
      --train_batch_size=1 \
      --guidance_scale=1 \
      --use_8bit_adam \
      --checkpointing_steps=100 \
      --gradient_accumulation_steps=4 \
      --optimizer="adamW" \
      --learning_rate=1e-4 \
      --lr_scheduler="constant" \
      --lr_warmup_steps=100 \
      --max_train_steps=1500 \
      --rank=4 \
      --seed="0" 

Usage

import torch
from diffusers import FluxPipeline

device = "cuda:0"

pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", torch_dtype=torch.bfloat16)

pipe.load_lora_weights("je-suis-tm/zoe_kravitz_lora_flux",
                       weight_name='pytorch_lora_weights.safetensors')


prompt = "a synthwave girl named Zoe Kravitz in foreground, buildings along miami beach with neon signs at night in background, retrowave synthwave aesthetics"

image = pipe(
  prompt=prompt,
  generator=torch.Generator(device=device).manual_seed(42),
  num_inference_steps=50, # 28 is a good trade-off
  guidance_scale=4,
  height=1024, 
  width=1024,
).images[0]

image.save("zoe_kravitz.png")

Trigger words

You should use Zoe Kravitz to trigger the image generation.

Download model

Download them in the Files & versions tab.