need testing
#1
by 3ndetz - opened
someone tested it? how it compares on the same seed for different quants? i didn't saw valuable differece
Hey man. I did quite some testing on your GGUF. Initially I couldn't notice any difference, in fact it was kind of slower (like 5%) compared to the original phr00t merge. But then I tried it in a different workflow than phr00t's and I managed to cut inference by 20% with GGUF Q4. I have a RTX 3080ti 16gb laptop, therefore FP8 shouldn't work for me.
Your GGUF saved me quite a lot of time and I cannot notice any noticeable quality degradation.
Honestly I don't know why I cannot see any difference in phr00ts workflow.
Gonna be testing Q5 soon.
thank you for your work (and phr00t's of course!)