File size: 1,757 Bytes
845ba57 248f0d3 845ba57 248f0d3 845ba57 248f0d3 845ba57 248f0d3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 | ---
license: mit
base_model: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
tags:
- quantized
- 4-bit
- int4
- awq
language:
- en
library_name: transformers
pipeline_tag: text-generation
---
# DeepSeek-R1-Distill-Qwen-1.5B-AWQ-INT4
INT4 weight-only quantization of [`deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B`](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B).
DeepSeek-R1 reasoning distilled into 1.5B Qwen, then INT4. Smallest reasoning model in the drawais lineup. Runs on a 4 GB consumer GPU.
| Property | Value |
|---|---|
| Base model | [deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B) |
| Quantization | INT4 weight-only |
| Approx. on-disk size | ~1.6 GB |
| Languages | English |
## Load (vLLM)
```bash
vllm serve drawais/DeepSeek-R1-Distill-Qwen-1.5B-AWQ-INT4 \
--max-model-len 32768 \
--gpu-memory-utilization 0.94
```
```python
from vllm import LLM, SamplingParams
llm = LLM(model="drawais/DeepSeek-R1-Distill-Qwen-1.5B-AWQ-INT4", max_model_len=32768)
print(llm.generate(["Hello!"], SamplingParams(max_tokens=128))[0].outputs[0].text)
```
## Footprint
~1.6 GB on disk. Recommended VRAM: enough headroom for KV cache.
## License & attribution
This artifact is a derivative work of [`deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B`](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B),
released by its original authors under the **MIT License**.
This artifact is distributed under the same license. The full license text is
included in [`LICENSE`](LICENSE), and required attribution is in [`NOTICE`](NOTICE).
License text: https://opensource.org/license/mit
Source model: https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
|