Update README.md
Browse files
README.md
CHANGED
|
@@ -21,4 +21,20 @@ tags:
|
|
| 21 |
|
| 22 |
## DistillSupra-0.2M
|
| 23 |
|
| 24 |
-
**DistillSupra-0.2M** is an ultra-compact causal language model with approximately **0.2 million parameters**, produced by knowledge distillation from [Supra-Mini-v4-2M](https://huggingface.co/SupraLabs/Supra-Mini-v4-2M).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
|
| 22 |
## DistillSupra-0.2M
|
| 23 |
|
| 24 |
+
**DistillSupra-0.2M** is an ultra-compact causal language model with approximately **0.2 million parameters**, produced by knowledge distillation from [Supra-Mini-v4-2M](https://huggingface.co/SupraLabs/Supra-Mini-v4-2M).
|
| 25 |
+
|
| 26 |
+
It was trained 500 steps for 30 minutes on a GTX 750 Ti 4GB using generated text from the teacher.
|
| 27 |
+
|
| 28 |
+
## Some outputs:
|
| 29 |
+
|
| 30 |
+
Prompt : Throughout history, great civilizations
|
| 31 |
+
|
| 32 |
+
Output: Throughout history, great civilizations to in, a be polrain for is with more the the be the for. of be of on (I.er The b M.A-R and or have that not is and the is this they, can for to to. is of a a, to ofs the for and the a. in the is to as of is that an that of and you the which is, the, for in be a are by’ of. and to a m
|
| 33 |
+
|
| 34 |
+
Prompt : The human brain is capable of
|
| 35 |
+
--------------------------------------------------
|
| 36 |
+
The human brain is capable ofs in an more that in a new can is the this the a of the pS, the a to the other in not it.
|
| 37 |
+
.. and with a to that be are of to for in of of ass. The be of the,.F-
|
| 38 |
+
s be the of dL
|
| 39 |
+
al. ins of be and of Sin: and or that a one that to and a a bFed, as
|
| 40 |
+
Ral., the, is a and as
|