16-bit quantization
#4
by semevas - opened
16-bit quantization?
The model uses bf16 by default. If you have an older GPU (like myself) and can't run bf16, try altering the dtype to "float16" in the model's config.json.
16-bit quantization?
The model uses bf16 by default. If you have an older GPU (like myself) and can't run bf16, try altering the dtype to "float16" in the model's config.json.