Qwen 2.5 72B Instruct Dynamic FP8
This is FP8 Dynamic Quantization (A8W8) for https://huggingface.co/Qwen/Qwen2.5-72B-Instruct, we use it for vLLM==0.8.5.post1 and above.
- Downloads last month
- 7,384
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support