TheBloke

/

stable-vicuna-13B-GPTQ

like 218

A 8bit version of the model would be helpful which can be loaded in 16GB of GPU VRAM

1. This is a 4bit GPTQ model. I could make an 8bit GPTQ but there's no point because we can already load HF models in 8bit using bitsandbytes
2. If you want 8bit, please use https://huggingface.co/TheBloke/stable-vicuna-13B-HF and specify load_in_8bit=True like I told you on Github

Sure will do that