Update README.md
Browse files
README.md
CHANGED
|
@@ -15,7 +15,7 @@ tags:
|
|
| 15 |
- rtx
|
| 16 |
- nvidia
|
| 17 |
- llama
|
| 18 |
-
-
|
| 19 |
- axionlab
|
| 20 |
library_name: transformers
|
| 21 |
---
|
|
@@ -24,7 +24,7 @@ library_name: transformers
|
|
| 24 |
|
| 25 |
So... have you ever seen a model that runs on a 3 dollars hardware? No? If no, Now you're seeing!
|
| 26 |
|
| 27 |
-
MicroSupra-1k is a bacteria base model(lol) trained on 300 million tokens of Fineweb-Edu for 3 epochs as the **first version** of our MicroSupra.
|
| 28 |
|
| 29 |
## Model Config
|
| 30 |
|
|
@@ -104,9 +104,9 @@ with torch.no_grad():
|
|
| 104 |
print("[*] Output:", tokenizer.decode(outputs[0], skip_special_tokens=True))
|
| 105 |
|
| 106 |
```
|
| 107 |
-
## Why SupraLabs
|
| 108 |
|
| 109 |
-
Because we are experimenting sizes, experiments(like 1Bit quant, distillation(NEW THINGS ARE COMING WITH DISTILLATION! GET TUNED!), pruning) all to better your experience! We are working
|
| 110 |
|
| 111 |
## Training guide
|
| 112 |
We trained MicroSupra on a GTX750 Ti 4GB in 1 Minute for 3 epochs.<br>
|
|
|
|
| 15 |
- rtx
|
| 16 |
- nvidia
|
| 17 |
- llama
|
| 18 |
+
- lh-tech
|
| 19 |
- axionlab
|
| 20 |
library_name: transformers
|
| 21 |
---
|
|
|
|
| 24 |
|
| 25 |
So... have you ever seen a model that runs on a 3 dollars hardware? No? If no, Now you're seeing!
|
| 26 |
|
| 27 |
+
MicroSupra-1k is a bacteria base model(lol) trained on 300 million tokens of Fineweb-Edu for 3 epochs as the **first version** of our MicroSupra series.
|
| 28 |
|
| 29 |
## Model Config
|
| 30 |
|
|
|
|
| 104 |
print("[*] Output:", tokenizer.decode(outputs[0], skip_special_tokens=True))
|
| 105 |
|
| 106 |
```
|
| 107 |
+
## Why did SupraLabs create this???
|
| 108 |
|
| 109 |
+
Because we are experimenting sizes, experiments(like 1Bit quant, distillation(NEW THINGS ARE COMING WITH DISTILLATION! GET TUNED!), pruning) all to better your experience! We are working on big things!
|
| 110 |
|
| 111 |
## Training guide
|
| 112 |
We trained MicroSupra on a GTX750 Ti 4GB in 1 Minute for 3 epochs.<br>
|