Snider Virgil commited on
Commit ·
6a5336a
1
Parent(s): da2ef7d
feat: add Q4_K_M + Q8_0 + BF16 gguf for Ollama / llama.cpp consumers
Browse filesConverted from model.safetensors via llama.cpp convert_hf_to_gguf.py
(bf16 intermediate) and llama-quantize (Q4_K_M + Q8_0).
Ollama pull paths:
ollama pull hf.co/LetheanNetwork/lemrd:Q4_K_M
ollama pull hf.co/LetheanNetwork/lemrd:Q8_0
ollama pull hf.co/LetheanNetwork/lemrd:BF16
Co-Authored-By: Virgil <virgil@lethean.io>
- .gitattributes +1 -0
- lemrd-bf16.gguf +3 -0
- lemrd-q4_k_m.gguf +3 -0
- lemrd-q8_0.gguf +3 -0
.gitattributes
CHANGED
|
@@ -36,3 +36,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 36 |
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
| 37 |
model-00001-of-00002.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 38 |
model-00002-of-00002.safetensors filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 36 |
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
| 37 |
model-00001-of-00002.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 38 |
model-00002-of-00002.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
*.gguf filter=lfs diff=lfs merge=lfs -text
|
lemrd-bf16.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8795756dde3f49a23e6c4287d20e4bbbd7be864e91fe53ef7eafbca81b96ea1d
|
| 3 |
+
size 61413183328
|
lemrd-q4_k_m.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5f7aae70a21ccae5d7cdc2df325f281438b4eb879c311bb30b5377a9e4b758dd
|
| 3 |
+
size 18687057760
|
lemrd-q8_0.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6ac66646c8f758cbe88c3885e0e853b35ae2c07b12b4bde7f4004f2e5e2a7007
|
| 3 |
+
size 32635670368
|