BachNgoH/flux2-klein-4b-overlap-ownership-lora-r32

FLUX.2 Klein 4B — Overlap-Ownership LoRA (Rank 32)

LoRA adapter fine-tuned on FLUX.2-klein-base-4B for count-preserving dense image generation with overlap-ownership attention zones.

Model Details

• Base model: black-forest-labs/FLUX.2-klein-base-4B
• LoRA rank: 32, alpha: 64
• Target modules: to_k, to_v, to_q, to_out.0, to_qkv_mlp_proj
• Trainable parameters: 23,592,960
• Training steps: 3000
• Training framework: PEFT + diffusers

Training Configuration

• Effective batch size: 8 (bs=1 x grad_accum=8)
• Learning rate: 1e-4
• LR scheduler: cosine with 200 warmup steps
• Optimizer: AdamW
• Precision: bf16
• Gradient clipping: 1.0
• Features: GLIGEN-style layout attention + AIBL loss + Overlap-Ownership zones

Overlap-Ownership Zones (Idea 7)

Three-zone classification for overlapping bboxes:

• Winner zone: pixels owned by one bbox (strong positive bias)
• Contested zone: pixels claimed by multiple bboxes (moderate positive bias)
• Loser zone: pixels occluded by other bboxes (reduced bias)

Parameters: omega_winner=0.5, omega_loser=0.3, omega_contested=0.5

Evaluation Results (1000 samples, val split)

Metric	Value	Baseline	Delta
Exact Match	15.32%	15.34%	-0.02%
±1 Tolerance	36.36%	—	—
±2 Tolerance	53.12%	—	—
MAE	3.891	4.22	-0.33
RMSE	5.993	8.12	-2.13

Per-Category (Top 10)

Category	MAE	Count	Avg GT
person	3.28	178	9.2
bottle	2.99	120	9.0
backpack	2.93	116	6.9
phone	3.14	93	6.4
book	9.36	64	14.0
chair	5.47	51	7.0
cup	4.04	45	14.0
bag	2.81	43	6.1
laptop	2.29	42	4.9
bench	4.44	39	4.4

Key Improvements over Rank 16

• MAE: 3.891 vs 4.285 (rank 16) — 9.2% improvement
• RMSE: 5.993 vs 6.844 (rank 16) — 12.4% improvement
• Doubled LoRA capacity (rank 32 vs 16) for better overlap-ownership pattern learning
• Higher learning rate (1e-4 vs 5e-5) with larger effective batch (8 vs 4)

Usage

from diffusers import Flux2KleinPipeline
from peft import PeftModel

pipe = Flux2KleinPipeline.from_pretrained("black-forest-labs/FLUX.2-klein-base-4B")
pipe.transformer = PeftModel.from_pretrained(pipe.transformer, "BachNgoH/flux2-klein-4b-overlap-ownership-lora-r32")

Framework Versions

• PEFT 0.18.1
• Diffusers
• PyTorch 2.x