Anyone up for making a FP8 for 4B embedding and reranker?
#25
by huggyfaceenjoyer - opened
Anyone up for making a FP8 for 4B embedding and reranker?
I'd love to know how to do it properly, I have been able to do it for the embedding, and it "seems" to work, but I am not 100% sure I did it correctly.