---
license: mit
base_model: deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
tags:
  - quantized
  - 4-bit
  - int4
  - reasoning
  - distill
language:
  - en
pipeline_tag: text-generation
---

# DeepSeek-R1-Distill-Qwen-7B-HQQ-INT4

INT4 quantization of [`deepseek-ai/DeepSeek-R1-Distill-Qwen-7B`](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B). Calibration-free companion to [`drawais/DeepSeek-R1-Distill-Qwen-7B-AWQ-INT4`](https://huggingface.co/drawais/DeepSeek-R1-Distill-Qwen-7B-AWQ-INT4).

## Footprint

| | |
|---|---|
| Source params | 7B (distilled from R1) |
| Quantized weights | ~5.3 GB on disk |
| Inference VRAM (incl. KV cache @ 32K context) | ~10 GB |

Best at native ≤32K context. For long-context use the AWQ companion via vLLM.

## Quick start

```python
from transformers import AutoTokenizer
from hqq.models.hf.base import AutoHQQHFModel

tok = AutoTokenizer.from_pretrained("drawais/DeepSeek-R1-Distill-Qwen-7B-HQQ-INT4")
model = AutoHQQHFModel.from_quantized("drawais/DeepSeek-R1-Distill-Qwen-7B-HQQ-INT4", device="cuda")
```

## License

MIT (DeepSeek-R1-Distill series). Underlying base model (Qwen2.5-Math-7B) is Apache 2.0.