Very cool. How did you quantize it?
Looks great, I'll try it out. It plays nice with all loras? Does it work for Klein9b? Z-Image Turbo? Thanks
i haven't tried comfy bathroom with anything outside ltx2.3 because its the only model I had the need to quantize so heavily, it is specifically coded for ltx2.3. but, it should theoretically work as a normal lora loader for other mdels if you just run default settings
when ltx2.3 dropped there wasn't any fp4's available, so i made my own, first one was like everyone elses, between 18-20gb and ran fine with lora's.
but i was curious how far i could push it and still get good results, around 13gb on gpu was the limit, anything beyond that, it degraded so bad.
the structure is block 0 and 47 are in bf16, block 1, 46, adaln, and a couple other things in fp8, the rest, all fp4.
this was smallest viable version i could get, but as I built it, wasn't testing with lora's, lora's were problematic.
spent days testing, turning on/off blocks in lora's, weighting blocks, etc, come to find out it was pretty simple, lora's were just over powering the low and high fp4 blocks because yes, bf16 overpowers fp4 like crazy, but it wasn't just as simple as lowering overall strength, because even at a strength of 0.01 on the lora, i had weird noise on stage 2 upscale because it was low step (3 steps), now you could fix it by running upscale to 9 steps, but who wants to do that?
so i tweaked, tested, found where each artifact was coming from, then built that node to compensate.
i do need to update it a little because I need another set of ramps for when stacking lora's because that noise starts to accumulate again. but the ramp to fix that is shown in the layout image in the "mirror", thats for 3+ more stacked loras.
i built the fp4 in comfy with comfy kitchen and bnb using a custom structure (i built 20 or so trying to find the best option, lol)