Quantized
Collection
A collection of quantized model I found. • 9 items • Updated
typhoon-si-med-thinking-4b-research-preview - GGUF
About
This repository contains GGUF weights for typhoon-ai/typhoon-si-med-thinking-4b-research-preview.
For a convenient overview and download list, visit our model page.
Usage
If you are unsure how to use GGUF files, refer to the llama.cpp documentation for more details.
Llama.cpp CLI
./llama-cli -m typhoon-si-med-thinking-4b-research-preview-q4_k_m.gguf -p "Hello!"
Provided Quants
(sorted by size, not necessarily quality)
| Link | Type | Size/GB | Notes |
|---|---|---|---|
| GGUF | q2_k | 1.67 | very low quality, for testing |
| GGUF | q3_k_m | 2.09 | |
| GGUF | q4_0 | 2.41 | |
| GGUF | q4_k_m | 2.53 | recommended, good balance |
| GGUF | q5_k_m | 2.94 | |
| GGUF | q8_0 | 4.37 | near-full precision |
Thanks
Special thanks to the llama.cpp team for their amazing work.
- Downloads last month
- 64
Hardware compatibility
Log In to add your hardware
2-bit
3-bit
4-bit
5-bit
8-bit
Model tree for sthaps/typhoon-si-med-thinking-4b-research-preview-gguf
Base model
Qwen/Qwen3-4B-Instruct-2507