SeifElden2342532/flux-lora-characters

FLUX.1-dev LoRA — Character Descriptions

A LoRA fine-tune of FLUX.1-dev trained on a character descriptions dataset. The model learns to generate detailed character portraits and illustrations from text descriptions.

---

Model Details

Base model	black-forest-labs/FLUX.1-dev
Training dataset	SeifElden2342532/characters_descriptions (2016 images)
LoRA rank	16
LoRA alpha	16
Training steps	2000
Resolution	512×512
Effective batch size	4 (batch 1 × grad accum 4)
Learning rate	1e-4 (cosine schedule)
GPU	NVIDIA H100 80GB
Training time	~10 minutes
Framework	Diffusers + PEFT

---

Usage

Install dependencies

pip install torch diffusers transformers accelerate peft safetensors

Run inference

import torch
from diffusers import FluxPipeline
from peft import PeftModel

# 1. Load base model
pipe = FluxPipeline.from_pretrained(
    "black-forest-labs/FLUX.1-dev",
    torch_dtype=torch.bfloat16,
    token="YOUR_HF_TOKEN",   # FLUX.1-dev requires access
).to("cuda")

# 2. Load LoRA weights
pipe.transformer = PeftModel.from_pretrained(
    pipe.transformer,
    "SeifElden2342532/flux-lora-characters",
)
pipe.transformer = pipe.transformer.merge_and_unload()

# 3. Generate
image = pipe(
    prompt               = "a portrait of a warrior character with armor",
    num_inference_steps  = 28,
    guidance_scale       = 3.5,
    generator            = torch.Generator("cuda").manual_seed(42),
).images[0]

image.save("character.png")

Example prompts

a portrait of a warrior character with heavy armor and a sword
a mage character with glowing robes and a magical staff
a rogue character with a hood and daggers
a healer character with white robes and a holy symbol
a portrait of a character with detailed facial features

---

Training Details

The model was fine-tuned using a custom FLUX LoRA training pipeline with:

• Flow matching loss with sigmoid timestep sampling
• bfloat16 mixed precision (no gradient scaler needed)
• Cosine LR schedule with 100 warmup steps
• Gradient accumulation over 4 steps
• LoRA applied to attention projections: to_q, to_k, to_v, to_out.0, add_q_proj, add_k_proj, add_v_proj
• Guidance tensor fixed at 3.5 (required for FLUX.1-dev distillation)

---

Limitations

• Works best with character/portrait prompts similar to training data
• Base model access required (black-forest-labs/FLUX.1-dev is gated)
• Best results at 28+ inference steps with guidance scale 3.5
• Resolution was trained at 512×512 — higher resolutions may vary in quality