Q8 mmproj is broken
#1
by RedAISkye - opened
f16 mmproj is working fine but Q8 mmproj is broken
"ggml_cuda_cpy: unsupported type combination (q8_0 to q8_0)"
I just tried it and it works, so this is more likely a bug/lack of support in your inference engine or usage problem.
mradermacher changed discussion status to closed
I just tried it and it works, so this is more likely a bug/lack of support in your inference engine or usage problem.
I'm using KoboldCpp.
This comment has been hidden (marked as Resolved)