DavidAU/Llama-3.2-3B-Instruct-NEO-SI-FI-GGUF · Could you quant a Q4_0_4

Could you quant a Q4_0_4_8 ?

by DocStrangeLoop - opened Oct 21, 2024

Hi there, I love this model, it performs absurdly well on mobile at Q3KM, could you also provide arm quantizations like Q4_0_4_8?

Owner Oct 22, 2024

@DocStrangeLoop These will be up in about an hour or so.

•

@DavidAU Looks like they uploaded for the 1B but not the 3B. Ty for the quick response.

@DavidAU Apreciate the work you're doing, still hoping to get a Q4_0_4_8 of the 3B as well.

Owner Nov 3, 2024

See what I can do ; don't have an ETA at the moment.

Owner Nov 9, 2024

@DocStrangeLoop Uploading (3 arms) now ; should be up in an hour or so.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment