Thanks to the author for their efforts, looking forward to the int4 version.
#1
by LawlietDream - opened
This was translated directly using a translation tool, so please excuse any grammatical errors. In short, I've been eagerly awaiting someone to create a V19 version of SVDQ, and you're the first. Thank you!
Still not there, very experimental it produces coherent outputs but still a way to go, the concept is validated tho so i'll be working on the next versions in the next days
An int4 version would be greatly appreciated, for those with older GPUs. Thanks regardless!
trying my best to fit this into 12gb vram, but calibration had been hard.
still researching this.