Carmenest commited on
Commit
af328fc
·
verified ·
1 Parent(s): 848ac23

docs: improve model card with quickstart, benchmarks, Apache-2.0

Browse files
Files changed (1) hide show
  1. README.md +55 -35
README.md CHANGED
@@ -1,21 +1,24 @@
1
  ---
2
  license: apache-2.0
3
  tags:
4
- - diffusion
5
- - masked-diffusion
6
- - dream
7
- - qwen2
8
- - gguf
9
- - diffuse-cpp
 
10
  base_model: Dream-org/Dream-v0-Instruct-7B
11
  pipeline_tag: text-generation
12
  ---
13
 
14
  # Dream-v0-Instruct-7B-GGUF
15
 
16
- GGUF quantizations of [Dream-org/Dream-v0-Instruct-7B](https://huggingface.co/Dream-org/Dream-v0-Instruct-7B) for use with [diffuse-cpp](https://github.com/iafiscal1212/diffuse-cpp), a CPU inference engine for Diffusion Language Models.
17
 
18
- Dream is a masked diffusion language model based on the Qwen2.5-7B backbone with bidirectional attention and Grouped Query Attention (GQA, 28 query heads / 4 KV heads).
 
 
19
 
20
  ## Available Quantizations
21
 
@@ -23,19 +26,37 @@ Dream is a masked diffusion language model based on the Qwen2.5-7B backbone with
23
  |------|------|------|-------------|
24
  | `dream-7b-f16.gguf` | F16 | ~15 GB | Full precision, best quality |
25
  | `dream-7b-q8_0.gguf` | Q8_0 | ~8.2 GB | 8-bit quantization, near-lossless |
26
- | `dream-7b-q4km.gguf` | Q4_K_M | ~5.0 GB | 4-bit mixed quantization, best quality/size ratio |
 
 
 
 
 
 
 
 
27
 
28
- **Recommended:** Q4_K_M for most users. Q8_0 if you have enough RAM and want minimal quality loss.
 
 
 
 
 
 
 
 
 
 
29
 
30
  ## Performance
31
 
32
- Benchmarked on diffuse-cpp with entropy_exit + inter-step KV cache, 12 threads, seed=42:
33
 
34
  | Prompt | tok/s | Steps | vs llama.cpp |
35
  |--------|-------|-------|-------------|
36
- | Capital of France? | 21.6 | 2 | 2.5x |
 
37
  | Translate to French | 14.3 | 6 | 1.7x |
38
- | 15 x 23? | 21.6 | 2 | 2.5x |
39
  | Translate to Spanish | 13.2 | 10 | 1.6x |
40
  | Python is_prime() | 8.2 | 7 | 1.0x |
41
  | Why sky blue? | 4.9 | 16 | 0.6x |
@@ -43,19 +64,20 @@ Benchmarked on diffuse-cpp with entropy_exit + inter-step KV cache, 12 threads,
43
  | Poem about ocean | 4.5 | 16 | 0.5x |
44
  | **Average** | **11.6** | | **1.4x** |
45
 
46
- - Easy prompts (factual, math): **14-22 tok/s** (1.6-2.5x faster than llama.cpp)
47
- - Hard prompts (creative, long-form): **4.5-4.9 tok/s**
48
- - llama.cpp baseline: 8.51 tok/s (Qwen2.5-7B-Instruct, Q4_K_M, same hardware)
49
 
50
- ## Usage
51
 
52
- ```bash
53
- # Download
54
- huggingface-cli download diffuse-cpp/Dream-v0-Instruct-7B-GGUF dream-7b-q4km.gguf
 
 
 
55
 
56
- # Run (requires diffuse-cpp v0.2.0+)
57
- ./diffuse-cli -m dream-7b-q4km.gguf -p "What is the capital of France?" -n 64 -s 16
58
- ```
59
 
60
  ## Model Details
61
 
@@ -63,30 +85,28 @@ huggingface-cli download diffuse-cpp/Dream-v0-Instruct-7B-GGUF dream-7b-q4km.ggu
63
  - **Parameters:** 7.62B
64
  - **Layers:** 28
65
  - **Hidden size:** 3584
66
- - **Attention:** GQA (28 query heads, 4 KV heads, head dim 128)
67
- - **FFN:** SwiGLU, intermediate size 18944
68
  - **Vocabulary:** 152,064 tokens
69
  - **RoPE theta:** 1,000,000
70
  - **Mask token ID:** 151666
71
- - **Training:** Masked diffusion on text, with autoregressive logit shift
72
 
73
  ## Conversion Details
74
 
75
- Converted from SafeTensors using `convert-dream.py` from diffuse-cpp:
76
- - 339 tensors total (255 weights + 84 QKV biases)
77
- - QKV biases kept at F32 in all quantizations
78
- - Edge layers (first/last) quantized to Q6_K in Q4_K_M scheme
79
 
80
  ## Citation
81
 
82
  ```bibtex
83
- @misc{dream2025,
84
- title={Dream 7B - Scalable Discrete Denoising Diffusion Models for Text Generation},
85
- author={Ye, Jiacheng and others},
86
- year={2025}
 
87
  }
88
  ```
89
 
90
  ## License
91
 
92
- Apache 2.0, following the original Dream model license.
 
1
  ---
2
  license: apache-2.0
3
  tags:
4
+ - diffusion
5
+ - dream
6
+ - gguf
7
+ - cpu-inference
8
+ - diffuse-cpp
9
+ language:
10
+ - en
11
  base_model: Dream-org/Dream-v0-Instruct-7B
12
  pipeline_tag: text-generation
13
  ---
14
 
15
  # Dream-v0-Instruct-7B-GGUF
16
 
17
+ GGUF quantizations of [Dream-org/Dream-v0-Instruct-7B](https://huggingface.co/Dream-org/Dream-v0-Instruct-7B) for use with [diffuse-cpp](https://github.com/iafiscal1212/diffuse-cpp), the first C++ inference engine for Diffusion Language Models.
18
 
19
+ Dream is a masked diffusion language model based on the Qwen2.5-7B backbone with Grouped Query Attention (GQA). It generates all tokens in parallel through iterative refinement, excelling at math and factual tasks.
20
+
21
+ **Dream correctly solves 15 x 23 = 345 in just 2 denoising steps at 21.6 tok/s — 2.5x faster than llama.cpp.**
22
 
23
  ## Available Quantizations
24
 
 
26
  |------|------|------|-------------|
27
  | `dream-7b-f16.gguf` | F16 | ~15 GB | Full precision, best quality |
28
  | `dream-7b-q8_0.gguf` | Q8_0 | ~8.2 GB | 8-bit quantization, near-lossless |
29
+ | `dream-7b-q4km.gguf` | Q4_K_M | ~5.0 GB | 4-bit mixed, best speed/quality ratio |
30
+
31
+ **Recommended:** Q4_K_M for most users.
32
+
33
+ ## Quick Start
34
+
35
+ ```bash
36
+ # Download
37
+ huggingface-cli download diffuse-cpp/Dream-v0-Instruct-7B-GGUF dream-7b-q4km.gguf
38
 
39
+ # Build diffuse-cpp (v0.2.0+)
40
+ git clone --recursive https://github.com/iafiscal1212/diffuse-cpp.git
41
+ cd diffuse-cpp
42
+ cmake -B build -DCMAKE_BUILD_TYPE=Release
43
+ cmake --build build -j$(nproc)
44
+
45
+ # Run
46
+ ./build/diffuse-cli -m ../dream-7b-q4km.gguf \
47
+ --tokens "151644,8948,198,2610,525,264,10950,17847,13,151645,198,151644,872,198,3838,374,220,868,1303,220,1419,30,151645,198,151644,77091,198" \
48
+ -n 64 -s 16 -t 12 --remasking entropy_exit
49
+ ```
50
 
51
  ## Performance
52
 
53
+ Benchmarked on AMD EPYC 4465P 12-Core, Q4_K_M, entropy_exit + inter-step cache, B=64:
54
 
55
  | Prompt | tok/s | Steps | vs llama.cpp |
56
  |--------|-------|-------|-------------|
57
+ | Capital of France? | **21.6** | 2 | 2.5x |
58
+ | 15 x 23? | **21.6** | 2 | 2.5x |
59
  | Translate to French | 14.3 | 6 | 1.7x |
 
60
  | Translate to Spanish | 13.2 | 10 | 1.6x |
61
  | Python is_prime() | 8.2 | 7 | 1.0x |
62
  | Why sky blue? | 4.9 | 16 | 0.6x |
 
64
  | Poem about ocean | 4.5 | 16 | 0.5x |
65
  | **Average** | **11.6** | | **1.4x** |
66
 
67
+ - Dream excels at **math and code** (converges in 2-7 steps)
68
+ - 5 of 8 prompts match or beat llama.cpp (8.51 tok/s baseline)
69
+ - llama.cpp baseline: Qwen2.5-7B-Instruct, Q4_K_M, same hardware
70
 
71
+ ## Dream vs LLaDA
72
 
73
+ | Strength | Dream-7B | LLaDA-8B |
74
+ |----------|----------|----------|
75
+ | Math/Arithmetic | 21.6 tok/s (2 steps) | 6.0 tok/s (16 steps) |
76
+ | Code generation | 8.2 tok/s (7 steps) | 4.5 tok/s (15 steps) |
77
+ | Translation | 13-14 tok/s | 23-28 tok/s |
78
+ | Creative writing | 4.5 tok/s | 5.0 tok/s |
79
 
80
+ **Use Dream for math, code, factual tasks. Use LLaDA for translation, conversation.**
 
 
81
 
82
  ## Model Details
83
 
 
85
  - **Parameters:** 7.62B
86
  - **Layers:** 28
87
  - **Hidden size:** 3584
88
+ - **Attention:** GQA (28 query / 4 KV heads)
89
+ - **FFN:** SwiGLU, intermediate 18944
90
  - **Vocabulary:** 152,064 tokens
91
  - **RoPE theta:** 1,000,000
92
  - **Mask token ID:** 151666
93
+ - **QKV biases:** Yes (kept at F32 in all quantizations)
94
 
95
  ## Conversion Details
96
 
97
+ 339 tensors (255 weights + 84 QKV biases). Converted with `convert-dream.py` from diffuse-cpp.
 
 
 
98
 
99
  ## Citation
100
 
101
  ```bibtex
102
+ @software{diffuse_cpp_2026,
103
+ title={diffuse-cpp: High-Performance Inference for Diffusion Language Models},
104
+ author={Carmen Esteban},
105
+ year={2026},
106
+ url={https://github.com/iafiscal1212/diffuse-cpp}
107
  }
108
  ```
109
 
110
  ## License
111
 
112
+ Apache 2.0