Snider Virgil commited on
Commit ·
47852c7
1
Parent(s): 983d4b7
feat: add gguf multi-quant set (migrated from lthn/lemmy-gguf)
Browse filesConsolidating the lemmy gguf quants into this repo to match the lemer
pattern — everything for llama.cpp/Ollama users in one place, mlx
users redirected to sibling -mlx repos. After this lands the separate
lthn/lemmy-gguf repo will be deleted.
Files (5 quants, ~127GB total):
lemmy-bf16.gguf 47 GB full precision reference
lemmy-q8_0.gguf 25 GB near-lossless
lemmy-q6_k.gguf 21 GB
lemmy-q5_k_m.gguf 18 GB
lemmy-q4_k_m.gguf 16 GB recommended for Ollama
Xet should dedupe most of the upload against the existing blobs
HF already has in lthn/lemmy-gguf.
Co-Authored-By: Virgil <virgil@lethean.io>
- lemmy-bf16.gguf +3 -0
- lemmy-q4_k_m.gguf +3 -0
- lemmy-q5_k_m.gguf +3 -0
- lemmy-q6_k.gguf +3 -0
- lemmy-q8_0.gguf +3 -0
lemmy-bf16.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c5516ca5e54fcf6a94c97df25a9b56fb7892d6fdb15cf1becdf40c5cb305c704
|
| 3 |
+
size 50505130048
|
lemmy-q4_k_m.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c4480dcd8afd99146ae9aee0ad8371b626d0c29c6ee2309607f290f9bbd0a01d
|
| 3 |
+
size 16796010560
|
lemmy-q5_k_m.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:120430df7f055c4a3dc73af2383d23e7420fd7d1210a09630dcc2b551534be0d
|
| 3 |
+
size 19132885056
|
lemmy-q6_k.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6b92e099f4ee8bb097d967504be105a53af5b1e4803ee2480fca4ef7019255d6
|
| 3 |
+
size 22638393920
|
lemmy-q8_0.gguf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c5b853555388c71de8b3aecb127ab51ba9fb5a97b026dc0e43d4cd8cb09787b3
|
| 3 |
+
size 26859853888
|