docs: improve model card with quickstart, benchmarks, Apache-2.0
Browse files
README.md
CHANGED
|
@@ -1,21 +1,24 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
tags:
|
| 4 |
-
- diffusion
|
| 5 |
-
-
|
| 6 |
-
-
|
| 7 |
-
-
|
| 8 |
-
-
|
| 9 |
-
|
|
|
|
| 10 |
base_model: Dream-org/Dream-v0-Instruct-7B
|
| 11 |
pipeline_tag: text-generation
|
| 12 |
---
|
| 13 |
|
| 14 |
# Dream-v0-Instruct-7B-GGUF
|
| 15 |
|
| 16 |
-
GGUF quantizations of [Dream-org/Dream-v0-Instruct-7B](https://huggingface.co/Dream-org/Dream-v0-Instruct-7B) for use with [diffuse-cpp](https://github.com/iafiscal1212/diffuse-cpp),
|
| 17 |
|
| 18 |
-
Dream is a masked diffusion language model based on the Qwen2.5-7B backbone with
|
|
|
|
|
|
|
| 19 |
|
| 20 |
## Available Quantizations
|
| 21 |
|
|
@@ -23,19 +26,37 @@ Dream is a masked diffusion language model based on the Qwen2.5-7B backbone with
|
|
| 23 |
|------|------|------|-------------|
|
| 24 |
| `dream-7b-f16.gguf` | F16 | ~15 GB | Full precision, best quality |
|
| 25 |
| `dream-7b-q8_0.gguf` | Q8_0 | ~8.2 GB | 8-bit quantization, near-lossless |
|
| 26 |
-
| `dream-7b-q4km.gguf` | Q4_K_M | ~5.0 GB | 4-bit mixed
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 27 |
|
| 28 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
|
| 30 |
## Performance
|
| 31 |
|
| 32 |
-
Benchmarked on
|
| 33 |
|
| 34 |
| Prompt | tok/s | Steps | vs llama.cpp |
|
| 35 |
|--------|-------|-------|-------------|
|
| 36 |
-
| Capital of France? | 21.6 | 2 | 2.5x |
|
|
|
|
| 37 |
| Translate to French | 14.3 | 6 | 1.7x |
|
| 38 |
-
| 15 x 23? | 21.6 | 2 | 2.5x |
|
| 39 |
| Translate to Spanish | 13.2 | 10 | 1.6x |
|
| 40 |
| Python is_prime() | 8.2 | 7 | 1.0x |
|
| 41 |
| Why sky blue? | 4.9 | 16 | 0.6x |
|
|
@@ -43,19 +64,20 @@ Benchmarked on diffuse-cpp with entropy_exit + inter-step KV cache, 12 threads,
|
|
| 43 |
| Poem about ocean | 4.5 | 16 | 0.5x |
|
| 44 |
| **Average** | **11.6** | | **1.4x** |
|
| 45 |
|
| 46 |
-
-
|
| 47 |
-
-
|
| 48 |
-
- llama.cpp baseline:
|
| 49 |
|
| 50 |
-
##
|
| 51 |
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
|
|
|
|
|
|
|
|
|
|
| 55 |
|
| 56 |
-
|
| 57 |
-
./diffuse-cli -m dream-7b-q4km.gguf -p "What is the capital of France?" -n 64 -s 16
|
| 58 |
-
```
|
| 59 |
|
| 60 |
## Model Details
|
| 61 |
|
|
@@ -63,30 +85,28 @@ huggingface-cli download diffuse-cpp/Dream-v0-Instruct-7B-GGUF dream-7b-q4km.ggu
|
|
| 63 |
- **Parameters:** 7.62B
|
| 64 |
- **Layers:** 28
|
| 65 |
- **Hidden size:** 3584
|
| 66 |
-
- **Attention:** GQA (28 query
|
| 67 |
-
- **FFN:** SwiGLU, intermediate
|
| 68 |
- **Vocabulary:** 152,064 tokens
|
| 69 |
- **RoPE theta:** 1,000,000
|
| 70 |
- **Mask token ID:** 151666
|
| 71 |
-
- **
|
| 72 |
|
| 73 |
## Conversion Details
|
| 74 |
|
| 75 |
-
|
| 76 |
-
- 339 tensors total (255 weights + 84 QKV biases)
|
| 77 |
-
- QKV biases kept at F32 in all quantizations
|
| 78 |
-
- Edge layers (first/last) quantized to Q6_K in Q4_K_M scheme
|
| 79 |
|
| 80 |
## Citation
|
| 81 |
|
| 82 |
```bibtex
|
| 83 |
-
@
|
| 84 |
-
title={
|
| 85 |
-
author={
|
| 86 |
-
year={
|
|
|
|
| 87 |
}
|
| 88 |
```
|
| 89 |
|
| 90 |
## License
|
| 91 |
|
| 92 |
-
Apache 2.0
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
tags:
|
| 4 |
+
- diffusion
|
| 5 |
+
- dream
|
| 6 |
+
- gguf
|
| 7 |
+
- cpu-inference
|
| 8 |
+
- diffuse-cpp
|
| 9 |
+
language:
|
| 10 |
+
- en
|
| 11 |
base_model: Dream-org/Dream-v0-Instruct-7B
|
| 12 |
pipeline_tag: text-generation
|
| 13 |
---
|
| 14 |
|
| 15 |
# Dream-v0-Instruct-7B-GGUF
|
| 16 |
|
| 17 |
+
GGUF quantizations of [Dream-org/Dream-v0-Instruct-7B](https://huggingface.co/Dream-org/Dream-v0-Instruct-7B) for use with [diffuse-cpp](https://github.com/iafiscal1212/diffuse-cpp), the first C++ inference engine for Diffusion Language Models.
|
| 18 |
|
| 19 |
+
Dream is a masked diffusion language model based on the Qwen2.5-7B backbone with Grouped Query Attention (GQA). It generates all tokens in parallel through iterative refinement, excelling at math and factual tasks.
|
| 20 |
+
|
| 21 |
+
**Dream correctly solves 15 x 23 = 345 in just 2 denoising steps at 21.6 tok/s — 2.5x faster than llama.cpp.**
|
| 22 |
|
| 23 |
## Available Quantizations
|
| 24 |
|
|
|
|
| 26 |
|------|------|------|-------------|
|
| 27 |
| `dream-7b-f16.gguf` | F16 | ~15 GB | Full precision, best quality |
|
| 28 |
| `dream-7b-q8_0.gguf` | Q8_0 | ~8.2 GB | 8-bit quantization, near-lossless |
|
| 29 |
+
| `dream-7b-q4km.gguf` | Q4_K_M | ~5.0 GB | 4-bit mixed, best speed/quality ratio |
|
| 30 |
+
|
| 31 |
+
**Recommended:** Q4_K_M for most users.
|
| 32 |
+
|
| 33 |
+
## Quick Start
|
| 34 |
+
|
| 35 |
+
```bash
|
| 36 |
+
# Download
|
| 37 |
+
huggingface-cli download diffuse-cpp/Dream-v0-Instruct-7B-GGUF dream-7b-q4km.gguf
|
| 38 |
|
| 39 |
+
# Build diffuse-cpp (v0.2.0+)
|
| 40 |
+
git clone --recursive https://github.com/iafiscal1212/diffuse-cpp.git
|
| 41 |
+
cd diffuse-cpp
|
| 42 |
+
cmake -B build -DCMAKE_BUILD_TYPE=Release
|
| 43 |
+
cmake --build build -j$(nproc)
|
| 44 |
+
|
| 45 |
+
# Run
|
| 46 |
+
./build/diffuse-cli -m ../dream-7b-q4km.gguf \
|
| 47 |
+
--tokens "151644,8948,198,2610,525,264,10950,17847,13,151645,198,151644,872,198,3838,374,220,868,1303,220,1419,30,151645,198,151644,77091,198" \
|
| 48 |
+
-n 64 -s 16 -t 12 --remasking entropy_exit
|
| 49 |
+
```
|
| 50 |
|
| 51 |
## Performance
|
| 52 |
|
| 53 |
+
Benchmarked on AMD EPYC 4465P 12-Core, Q4_K_M, entropy_exit + inter-step cache, B=64:
|
| 54 |
|
| 55 |
| Prompt | tok/s | Steps | vs llama.cpp |
|
| 56 |
|--------|-------|-------|-------------|
|
| 57 |
+
| Capital of France? | **21.6** | 2 | 2.5x |
|
| 58 |
+
| 15 x 23? | **21.6** | 2 | 2.5x |
|
| 59 |
| Translate to French | 14.3 | 6 | 1.7x |
|
|
|
|
| 60 |
| Translate to Spanish | 13.2 | 10 | 1.6x |
|
| 61 |
| Python is_prime() | 8.2 | 7 | 1.0x |
|
| 62 |
| Why sky blue? | 4.9 | 16 | 0.6x |
|
|
|
|
| 64 |
| Poem about ocean | 4.5 | 16 | 0.5x |
|
| 65 |
| **Average** | **11.6** | | **1.4x** |
|
| 66 |
|
| 67 |
+
- Dream excels at **math and code** (converges in 2-7 steps)
|
| 68 |
+
- 5 of 8 prompts match or beat llama.cpp (8.51 tok/s baseline)
|
| 69 |
+
- llama.cpp baseline: Qwen2.5-7B-Instruct, Q4_K_M, same hardware
|
| 70 |
|
| 71 |
+
## Dream vs LLaDA
|
| 72 |
|
| 73 |
+
| Strength | Dream-7B | LLaDA-8B |
|
| 74 |
+
|----------|----------|----------|
|
| 75 |
+
| Math/Arithmetic | 21.6 tok/s (2 steps) | 6.0 tok/s (16 steps) |
|
| 76 |
+
| Code generation | 8.2 tok/s (7 steps) | 4.5 tok/s (15 steps) |
|
| 77 |
+
| Translation | 13-14 tok/s | 23-28 tok/s |
|
| 78 |
+
| Creative writing | 4.5 tok/s | 5.0 tok/s |
|
| 79 |
|
| 80 |
+
**Use Dream for math, code, factual tasks. Use LLaDA for translation, conversation.**
|
|
|
|
|
|
|
| 81 |
|
| 82 |
## Model Details
|
| 83 |
|
|
|
|
| 85 |
- **Parameters:** 7.62B
|
| 86 |
- **Layers:** 28
|
| 87 |
- **Hidden size:** 3584
|
| 88 |
+
- **Attention:** GQA (28 query / 4 KV heads)
|
| 89 |
+
- **FFN:** SwiGLU, intermediate 18944
|
| 90 |
- **Vocabulary:** 152,064 tokens
|
| 91 |
- **RoPE theta:** 1,000,000
|
| 92 |
- **Mask token ID:** 151666
|
| 93 |
+
- **QKV biases:** Yes (kept at F32 in all quantizations)
|
| 94 |
|
| 95 |
## Conversion Details
|
| 96 |
|
| 97 |
+
339 tensors (255 weights + 84 QKV biases). Converted with `convert-dream.py` from diffuse-cpp.
|
|
|
|
|
|
|
|
|
|
| 98 |
|
| 99 |
## Citation
|
| 100 |
|
| 101 |
```bibtex
|
| 102 |
+
@software{diffuse_cpp_2026,
|
| 103 |
+
title={diffuse-cpp: High-Performance Inference for Diffusion Language Models},
|
| 104 |
+
author={Carmen Esteban},
|
| 105 |
+
year={2026},
|
| 106 |
+
url={https://github.com/iafiscal1212/diffuse-cpp}
|
| 107 |
}
|
| 108 |
```
|
| 109 |
|
| 110 |
## License
|
| 111 |
|
| 112 |
+
Apache 2.0
|