needle-1M-bench + Qwen3 quantizations
Collection
Long-context faithfulness benchmark + audit-friendly Qwen3 quantized releases. Outputs ship; inputs are auditable. • 23 items • Updated
Qwen3-4B-HQQ-INT4
INT4 quantization of Qwen/Qwen3-4B. Calibration-free companion to drawais/Qwen3-4B-AWQ-INT4.
Footprint
| Source params | 4B |
| Quantized weights | ~2.6 GB on disk |
| Inference VRAM (incl. KV cache @ 32K context) | ~6 GB |
The smallest tier in the kit. Recommended at native ≤40K context. For long-context use the AWQ companion via vLLM.
Quick start
from transformers import AutoTokenizer
from hqq.models.hf.base import AutoHQQHFModel
tok = AutoTokenizer.from_pretrained("drawais/Qwen3-4B-HQQ-INT4")
model = AutoHQQHFModel.from_quantized("drawais/Qwen3-4B-HQQ-INT4", device="cuda")
License
Apache 2.0 (inherits from base model).
- Downloads last month
- 13