Hugging Face's logo

bluezabu
/
klimt-lora

Text-to-Image
Diffusers
flux
flux-diffusers
image-to-image
simpletuner
Not-For-All-Audiences
lora
template:sd-lora
standard
Model card Files
xet
 Community

klimt-lora

This is a standard PEFT LoRA derived from black-forest-labs/FLUX.1-dev.

No validation prompt was used during training.

None

Validation settings

  • CFG: 3.0
  • CFG Rescale: 0.0
  • Steps: 20
  • Sampler: FlowMatchEulerDiscreteScheduler
  • Seed: 1337
  • Resolution: 1024x1024
  • Skip-layer guidance:

Note: The validation settings are not necessarily the same as the training settings.

You can find some example images in the following gallery:

Prompt
unconditional (blank prompt)
Negative Prompt
blurry, cropped, ugly
Prompt
The painting depicts an old stone building partially covered in ivy with several windows visible along its facade, situated beside calm water reflecting parts of the structure.
Negative Prompt
blurry, cropped, ugly
Prompt
The painting depicts an expansive sunflower with large green leaves surrounded by colorful flowers in various shades including reds, blues, whites, yellows, and oranges against a textured background filled with smaller white dots that resemble stars or snowflakes scattered across it.
Negative Prompt
blurry, cropped, ugly
Prompt
The painting depicts two figures in an abstract setting with intricate patterns and vibrant colors dominating the background. The left figure appears to be elderly and thin, while the right figure holds a child close, both standing against a backdrop filled with geometric shapes and circular motifs.
Negative Prompt
blurry, cropped, ugly
Prompt
The painting depicts two women in vibrant attire surrounded by ornate birds against an abstract red background adorned with floral patterns.
Negative Prompt
blurry, cropped, ugly
Prompt
The painting depicts an Italian village with white buildings nestled among lush greenery and tall cypress trees along a riverbank under a clear sky.
Negative Prompt
blurry, cropped, ugly
Prompt
The painting depicts a stone bridge crossing a quiet river, with weeping willow trees hanging over the water and ducks swimming near the bank.
Negative Prompt
blurry, cropped, ugly
Prompt
The painting depicts a marketplace in a small European town, with vendors selling fruits and flowers under striped awnings and people walking on cobblestone streets.
Negative Prompt
blurry, cropped, ugly
Prompt
The painting depicts a wide field of lavender in full bloom stretching toward distant hills, with a single farmhouse and a dirt path cutting through the rows of flowers.
Negative Prompt
blurry, cropped, ugly
Prompt
The painting depicts a harbor at sunset, with sailboats moored along the wooden docks and reflections of orange and pink light dancing across the water.
Negative Prompt
blurry, cropped, ugly
Prompt
The painting depicts a young girl sitting on a wooden porch shelling peas into a bowl, with a dog lying beside her and a garden blooming in the background.
Negative Prompt
blurry, cropped, ugly

The text encoder was not trained. You may reuse the base model text encoder for inference.

Training settings

  • Training epochs: 1

  • Training steps: 4000

  • Learning rate: 8e-05

    • Learning rate schedule: polynomial
    • Warmup steps: 100

  • Max grad value: 0.1

  • Effective batch size: 2

    • Micro-batch size: 2
    • Gradient accumulation steps: 1
    • Number of GPUs: 1

  • Gradient checkpointing: True

  • Prediction type: flow-matching (extra parameters=['shift=3', 'flux_guidance_mode=constant', 'flux_guidance_value=1.0', 'flow_matching_loss=compatible', 'flux_lora_target=all'])

  • Optimizer: adamw_bf16

  • Trainable parameter precision: Pure BF16

  • Base model precision: int8-quanto

  • Caption dropout probability: 3.0%

  • LoRA Rank: 64

  • LoRA Alpha: None

  • LoRA Dropout: 0.1

  • LoRA initialisation style: default

Datasets

klimt-256

  • Repeats: 10
  • Total number of images: 94
  • Total number of aspect buckets: 1
  • Resolution: 0.065536 megapixels
  • Cropped: False
  • Crop style: None
  • Crop aspect: None
  • Used for regularisation data: No

klimt-crop-256

  • Repeats: 10
  • Total number of images: 94
  • Total number of aspect buckets: 1
  • Resolution: 0.065536 megapixels
  • Cropped: True
  • Crop style: center
  • Crop aspect: square
  • Used for regularisation data: No

klimt-512

  • Repeats: 4
  • Total number of images: 94
  • Total number of aspect buckets: 13
  • Resolution: 0.262144 megapixels
  • Cropped: False
  • Crop style: None
  • Crop aspect: None
  • Used for regularisation data: No

klimt-crop-512

  • Repeats: 4
  • Total number of images: 94
  • Total number of aspect buckets: 1
  • Resolution: 0.262144 megapixels
  • Cropped: True
  • Crop style: center
  • Crop aspect: square
  • Used for regularisation data: No

klimt-768

  • Repeats: 3
  • Total number of images: 92
  • Total number of aspect buckets: 17
  • Resolution: 0.589824 megapixels
  • Cropped: False
  • Crop style: None
  • Crop aspect: None
  • Used for regularisation data: No

klimt-crop-768

  • Repeats: 3
  • Total number of images: 88
  • Total number of aspect buckets: 1
  • Resolution: 0.589824 megapixels
  • Cropped: True
  • Crop style: center
  • Crop aspect: square
  • Used for regularisation data: No

klimt-1024

  • Repeats: 2
  • Total number of images: 86
  • Total number of aspect buckets: 18
  • Resolution: 1.048576 megapixels
  • Cropped: False
  • Crop style: None
  • Crop aspect: None
  • Used for regularisation data: No

klimt-crop-1024

  • Repeats: 2
  • Total number of images: 84
  • Total number of aspect buckets: 1
  • Resolution: 1.048576 megapixels
  • Cropped: True
  • Crop style: center
  • Crop aspect: square
  • Used for regularisation data: No

klimt-1440

  • Repeats: 0
  • Total number of images: 64
  • Total number of aspect buckets: 9
  • Resolution: 2.0736 megapixels
  • Cropped: False
  • Crop style: None
  • Crop aspect: None
  • Used for regularisation data: No

klimt-crop-1440

  • Repeats: 0
  • Total number of images: 75
  • Total number of aspect buckets: 1
  • Resolution: 2.0736 megapixels
  • Cropped: True
  • Crop style: center
  • Crop aspect: square
  • Used for regularisation data: No

Inference 

import torch
from diffusers import DiffusionPipeline

model_id = 'black-forest-labs/FLUX.1-dev'
adapter_id = 'bluezabu/klimt-lora'
pipeline = DiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.bfloat16) # loading directly in bf16
pipeline.load_lora_weights(adapter_id)

prompt = "An astronaut is riding a horse through the jungles of Thailand."


## Optional: quantise the model to save on vram.
## Note: The model was quantised during training, and so it is recommended to do the same during inference time.
from optimum.quanto import quantize, freeze, qint8
quantize(pipeline.transformer, weights=qint8)
freeze(pipeline.transformer)
    
pipeline.to('cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu') # the pipeline is already in its target precision level
model_output = pipeline(
    prompt=prompt,
    num_inference_steps=20,
    generator=torch.Generator(device='cuda' if torch.cuda.is_available() else 'mps' if torch.backends.mps.is_available() else 'cpu').manual_seed(1337),
    width=1024,
    height=1024,
    guidance_scale=3.0,
).images[0]

model_output.save("output.png", format="PNG")
Downloads last month
3
Inference Providers NEW
Text-to-Image
Examples

Model tree for bluezabu/klimt-lora

Base model

black-forest-labs/FLUX.1-dev
Adapter
(36804)
this model