knoxel commited on
Commit
aec068f
Β·
verified Β·
1 Parent(s): 541bc33

Upload README.md

Browse files
Files changed (1) hide show
  1. README.md +49 -5
README.md CHANGED
@@ -1,12 +1,56 @@
1
  ---
2
- title: Bitnet B158 Cpu Explorer
3
- emoji: πŸ‘€
4
- colorFrom: blue
5
- colorTo: pink
6
  sdk: gradio
7
  sdk_version: 6.14.0
8
  app_file: app.py
9
  pinned: false
 
 
 
 
 
 
 
 
 
10
  ---
11
 
12
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: "🧬 BitNet b1.58 2B4T β€” CPU-Only Inference Explorer"
3
+ emoji: 🧬
4
+ colorFrom: indigo
5
+ colorTo: blue
6
  sdk: gradio
7
  sdk_version: 6.14.0
8
  app_file: app.py
9
  pinned: false
10
+ models:
11
+ - microsoft/bitnet-b1.58-2B-4T
12
+ tags:
13
+ - bitnet
14
+ - 1-bit
15
+ - cpu-inference
16
+ - ternary-weights
17
+ - efficient-inference
18
+ short_description: "Chat with Microsoft's 1-bit LLM on CPU β€” no GPU needed"
19
  ---
20
 
21
+ # 🧬 BitNet b1.58 2B4T β€” CPU-Only Inference Explorer
22
+
23
+ An interactive demo of **Microsoft Research's first open-source native 1-bit Large Language Model**.
24
+
25
+ ## What makes this special?
26
+
27
+ | Feature | Detail |
28
+ |---|---|
29
+ | **Weights** | Ternary {-1, 0, +1} β€” just 1.58 bits per weight |
30
+ | **Memory** | 0.4 GB (non-embedding) β€” **5-13Γ— less** than comparable models |
31
+ | **Energy** | 0.028J per token β€” **6-9Γ— less** than FP16 models |
32
+ | **Quality** | 54.2% avg benchmark β€” competitive with Qwen2.5 1.5B (55.2%) |
33
+ | **Training** | Trained from scratch on 4T tokens (not post-training quantized) |
34
+
35
+ ## Key insight
36
+
37
+ Since weights are only -1, 0, or +1, matrix multiplication becomes pure **addition/subtraction**. No floating-point multiplies needed β€” this is why CPUs can run BitNet efficiently.
38
+
39
+ ## Demo features
40
+
41
+ - πŸ’¬ **Chat** β€” Streaming conversation with live tokens/sec stats
42
+ - πŸ“Š **Benchmark** β€” Single-shot generation with memory & speed metrics
43
+ - πŸ“ˆ **Paper Results** β€” Published benchmark comparison table
44
+ - πŸ—οΈ **Architecture** β€” Visual explainer of how BitNet b1.58 differs from standard Transformers
45
+ - βš™οΈ **System** β€” Live hardware & memory stats
46
+
47
+ ## ⚠️ Performance note
48
+
49
+ This demo uses the `transformers` library, which does **not** include the specialized `bitnet.cpp` kernels. For the paper's reported CPU latency (29ms/token), use [bitnet.cpp](https://github.com/microsoft/BitNet) with the [GGUF weights](https://huggingface.co/microsoft/bitnet-b1.58-2B-4T-gguf).
50
+
51
+ ## References
52
+
53
+ - πŸ“„ [Technical Report](https://arxiv.org/abs/2504.12285)
54
+ - πŸ€— [Model Weights](https://huggingface.co/microsoft/bitnet-b1.58-2B-4T)
55
+ - πŸ’» [bitnet.cpp](https://github.com/microsoft/BitNet) (38K+ ⭐)
56
+ - πŸ“¦ [GGUF Weights](https://huggingface.co/microsoft/bitnet-b1.58-2B-4T-gguf)