---
license: apache-2.0
base_model: lablab-ai-amd-developer-hackathon/CyberSecQwen-4B
tags:
- qwen3
- cybersecurity
- cti
- cwe-classification
- vulnerability-analysis
- gguf
- q4_k_m
- 4-bit
- quantized
---

# CyberSecQwen-4B-GGUF

GGUF Q4_K_M quantized version of [CyberSecQwen-4B](https://huggingface.co/lablab-ai-amd-developer-hackathon/CyberSecQwen-4B).

## Quantization

| Parameter | Value |
|---|---|
| Method | GGUF Q4_K_M (llama.cpp) |
| Weight precision | 4-bit (Q4_K_M = 4-bit block-scaled with k-quant importance) |
| Quantization tool | llama.cpp (build from master) |
| Conversion tool | convert_hf_to_gguf.py |
| Quantization hardware | Modal A10G |
| File | cybersecqwen-4b-Q4_K_M.gguf (2.5 GB) |

## CTI-Bench Evaluation

Evaluated under the [Foundation-Sec-8B protocol](https://arxiv.org/abs/2504.21039):
- Temperature 0.3, max_tokens 512, concurrency 8
- 5 independent trials, zero-shot (no system prompt)
- llama.cpp server on Modal L4 GPU

| Task | GGUF Q4_K_M | AWQ 4-bit | FP16 Reference |
|---|---|---|---|
| CTI-MCQ (2,500 items) | 0.5368 ± 0.0048 | **0.5921 ± 0.0083** | 0.5868 ± 0.0029 |
| CTI-RCM (1,000 items) | **0.6254 ± 0.0063** | 0.5814 ± 0.0025 | 0.6664 ± 0.0023 |

**Key findings:**
- **CTI-RCM** (CVE→CWE classification): GGUF Q4_K_M is the best quantized variant (-4.1 pts vs FP16). Superior to AWQ 4-bit by +4.4 points.
- **CTI-MCQ** (CTI knowledge): AWQ 4-bit performs better than GGUF for multiple-choice questions.
- GGUF preserves task-specific classification accuracy better due to block-wise k-quant importance scaling.

## Trial results

### CTI-MCQ
| Trial | Seed | Accuracy |
|---|---|---|
| 1 | 42 | 0.5420 |
| 2 | 43 | 0.5280 |
| 3 | 44 | 0.5360 |
| 4 | 45 | 0.5392 |
| 5 | 46 | 0.5388 |

### CTI-MCQ

| Trial | Seed | Accuracy |
|---|---|---|
| 1 | 42 | 0.6270 |
| 2 | 43 | 0.6300 |
| 3 | 44 | 0.6270 |
| 4 | 45 | 0.6300 |
| 5 | 46 | 0.6130 |

## Quantization variants

| Variant | CTI-MCQ | CTI-RCM | Size | Engine |
|---|---|---|---|---|
| [AWQ 4-bit](https://huggingface.co/ree2raz/CyberSecQwen-4B-AWQ) | 0.5921 | 0.5814 | 2.7 GB | vLLM |
| [GGUF Q4_K_M](https://huggingface.co/ree2raz/CyberSecQwen-4B-GGUF) | 0.5368 | 0.6254 | 2.5 GB | llama.cpp |

Choose GGUF for vulnerability classification, AWQ for MCQ/general chat.

## Usage with llama.cpp

```bash
# Download
wget https://huggingface.co/ree2raz/CyberSecQwen-4B-GGUF/resolve/main/cybersecqwen-4b-Q4_K_M.gguf

# Serve
./llama-server -m cybersecqwen-4b-Q4_K_M.gguf --host 0.0.0.0 --port 8080 -ngl 99 -c 4096
```

## Model Size

| Format | Size |
|---|---|
| Original FP16 | ~8 GB |
| GGUF Q4_K_M | ~2.5 GB |

## Citation

```bibtex
@misc{{cybersecqwen2026,
  title  = {{CyberSecQwen-4B: A Compact CTI Specialist}},
  author = {{Mulia, Samuel}},
  year   = {{2026}},
  url    = {{https://huggingface.co/athena129/CyberSecQwen-4B}}
}}
```

## Evaluation Infrastructure

[GitHub repository](https://github.com/ree2raz/cyberSecQwen_4b_4bit) — Modal scripts for quantization + evaluation.