iproskurina/Mistral-7B-v0.3-GPTQ-4bit-g128
Text Generation • 7B • Updated • 28
Quantized LLMs with GPTQ
LLMs quantized with GPTQ
-
-
iproskurina/bloom-7b1-GPTQ-4bit-g128
Text Generation • 3B • Updated • 16 • 2 -
iproskurina/bloom-1b7-GPTQ-4bit-g128
Text Generation • 1B • Updated • 7 -
iproskurina/bloom-3b-GPTQ-4bit-g128
Text Generation • 2B • Updated • 4 -
iproskurina/bloom-560m-GPTQ-4bit-g128
Text Generation • 0.6B • Updated • 4 -
iproskurina/bloom-1b1-GPTQ-4bit-g128
Text Generation • 0.9B • Updated • 11 -
iproskurina/opt-2.7b-GPTQ-4bit-g128
Text Generation • 0.6B • Updated • 18 -
iproskurina/opt-13b-GPTQ-4bit-g128
Text Generation • 2B • Updated • 7 -
iproskurina/opt-6.7b-GPTQ-4bit-g128
Text Generation • 1B • Updated • 7 -
iproskurina/opt-125m-GPTQ-4bit-g128
Text Generation • Updated • 4 -
iproskurina/opt-350m-GPTQ-4bit-g128
Text Generation • 95.6M • Updated • 9 -
iproskurina/opt-1.3b-GPTQ-4bit-g128
Text Generation • 0.4B • Updated • 10 -
iproskurina/Mistral-7B-v0.1-GPTQ-8bit-g128
Text Generation • 2B • Updated • 3 -
iproskurina/Mistral-7B-v0.3-GPTQ-8bit-g128
Text Generation • 7B • Updated • 2 -
iproskurina/Mistral-7B-v0.1-GPTQ-3bit-g64
Text Generation • 1B • Updated • 2 -
iproskurina/Mistral-7B-v0.1-GPTQ-8bit-g64
Text Generation • 2B • Updated • 1 -
iproskurina/Mistral-7B-v0.1-GPTQ-4bit-g128
Text Generation • 1B • Updated • 3 -
iproskurina/Mistral-7B-v0.1-GPTQ-3bit-g128
Text Generation • 1.0B • Updated • 3 -
TheBloke/Mistral-7B-Instruct-v0.1-GPTQ
Text Generation • 7B • Updated • 347 • 84 -
TheBloke/Mistral-7B-Instruct-v0.2-GPTQ
Text Generation • 7B • Updated • 12.3k • 55 -
TheBloke/bloomz-176B-GPTQ
Text Generation • Updated • 26 • 19 -
TheBloke/BLOOMChat-176B-v1-GPTQ
Text Generation • Updated • 8 • 31 -
TheBloke/Llama-2-13B-chat-GPTQ
Text Generation • 13B • Updated • 1.45k • 363 -
When Quantization Affects Confidence of Large Language Models?
Paper • 2405.00632 • Published