giannisan/light-fantasy-flux2-klein-lora

Light Fantasy — FLUX.2 Klein 4B LoRA

A LoRA fine-tune of FLUX.2-klein-4B trained on 232 fantasy paintings in a luminous, painterly style with vibrant colors — castles, knights, enchanted landscapes, flowers, and magical atmospheres.

Quick Start

import torch
from diffusers import Flux2KleinPipeline, BitsAndBytesConfig, Flux2Transformer2DModel

# Load base model with 4-bit quantization (fits in 16GB VRAM)
nf4_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
)
transformer = Flux2Transformer2DModel.from_pretrained(
    "black-forest-labs/FLUX.2-klein-4B",
    subfolder="transformer",
    quantization_config=nf4_config,
    torch_dtype=torch.bfloat16,
)
pipe = Flux2KleinPipeline.from_pretrained(
    "black-forest-labs/FLUX.2-klein-4B",
    transformer=transformer,
    torch_dtype=torch.bfloat16,
)

# Load LoRA weights
pipe.load_lora_weights("giannisan/light-fantasy-flux2-klein-lora")
pipe.enable_model_cpu_offload()

# Fix VAE dtype mismatch
_orig_decode = pipe.vae._decode
def _patched_decode(z, *args, **kwargs):
    return _orig_decode(z.to(pipe.vae.dtype), *args, **kwargs)
pipe.vae._decode = _patched_decode

# Generate
image = pipe(
    prompt="light_fantasy, a detailed fantasy painting of a grand castle on a mountainside with waterfalls and wildflowers",
    height=512,
    width=512,
    num_inference_steps=50,
    guidance_scale=1.0,
    generator=torch.Generator("cpu").manual_seed(42),
).images[0]
image.save("output.png")

Trigger Word

Use light_fantasy at the start of your prompt to activate the style. For best results, follow it with a descriptive scene:

light_fantasy, a detailed fantasy painting of [your scene description]

Important Notes

• Use 50 inference steps for best quality. The LoRA was trained on the step-distilled model, which disrupts the 4-step distillation. At 4 steps images will appear blurry; at 50 steps they are sharp and detailed.
• guidance_scale=1.0 — this is a distilled model, CFG guidance is ignored.
• Resolution: Trained at 512×512. Works well at 512×768 for landscape compositions.
• VRAM: Runs on 16GB GPUs with NF4 quantization + CPU offload.

Example Prompts

Prompt	Style
light_fantasy, a detailed fantasy painting of a medieval harbor town at sunset with tall ships	In-distribution
light_fantasy, a detailed fantasy painting of a dragon sleeping on gold in a crystal cavern	Out-of-distribution (new subject, trained style)
light_fantasy, a detailed fantasy painting of a cozy wizard's library with floating books	Out-of-distribution

Training Details

Parameter	Value
Base model	FLUX.2-klein-4B
Method	DreamBooth LoRA with QLoRA (NF4 quantization)
Dataset	232 fantasy painting images with per-image BLIP captions
Resolution	512×512
LoRA rank	16
Learning rate	1e-4 (constant, 100 warmup steps)
Training steps	2000
Batch size	1 (gradient accumulation 4)
Optimizer	AdamW 8-bit
Mixed precision	bf16
Final loss	0.969
Hardware	NVIDIA RTX 4060 Ti 16GB
Training time	~3 hours

Dataset

Trained on giannisan/light-fantasy-dataset — 232 fantasy paintings auto-captioned with BLIP (Salesforce/blip-image-captioning-large). Each caption starts with the light_fantasy trigger word.

License

This LoRA inherits the license from the base model. See FLUX.2 License.