3.0bpw Versions
#1
by Amajiro - opened
I run a 3090 and a 3060 together, allowing me to run your other 70b 3.0bpw versions with 8192 context and they’re great. Any chance of a 3.0bpw version of this, or even a little bit higher? 36Gb VRAM could stretch to maybe 3.2.
And there it is.
Amajiro changed discussion status to closed