ParoQuant
Collection
Pairwise Rotation Quantization for Efficient Reasoning LLM Inference • 4 items • Updated
Jeethu/gemma-4-E4B-it-PARO
Pairwise Rotation Quantization for Efficient Reasoning LLM Inference
ParoQuant is the state-of-the-art INT4 quantization for LLMs. It closes the accuracy gap with FP16 while running at near-AWQ speed. Supports NVIDIA GPUs (vLLM, Transformers) and Apple Silicon (MLX). For more information, see https://github.com/z-lab/paroquant.
Jeethu/gemma-4-E4B-it-PARO is a 4-bit google/gemma-4-E4B-it quantized with ParoQuant.
- Downloads last month
- 15
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support