Works well converted to GGUF Q8_0
#1
by Joseph717171 - opened
Just tried your FP8-Dynamic of Llama-3.1-8B-Instruct. Its good. I converted it to GGUF Q8_0; it ran great. Can't wait to try the FP8-Block version. 😋
Just tried your FP8-Dynamic of Llama-3.1-8B-Instruct. Its good. I converted it to GGUF Q8_0; it ran great. Can't wait to try the FP8-Block version. 😋