great stuff! and a question!
#1
by erosdiffusion - opened
fist of all, thanks for converting to int8!
i can successfully run this model on 10gb + 32gb ram
using https://github.com/BobJohnson24/ComfyUI-INT8-Fast , and the api clip encoding from ltx package.
You're welcome. It's been tested in all INT8 nodes (QuantOps and the W8A8). Glad it worked for you. You can load the Gemma 12b model and the distilled model together in the dual int8 clip loader.