AxionLab-official commited on
Commit
cb4be93
·
verified ·
1 Parent(s): 3bcc458

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +17 -1
README.md CHANGED
@@ -21,4 +21,20 @@ tags:
21
 
22
  ## DistillSupra-0.2M
23
 
24
- **DistillSupra-0.2M** is an ultra-compact causal language model with approximately **0.2 million parameters**, produced by knowledge distillation from [Supra-Mini-v4-2M](https://huggingface.co/SupraLabs/Supra-Mini-v4-2M).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
21
 
22
  ## DistillSupra-0.2M
23
 
24
+ **DistillSupra-0.2M** is an ultra-compact causal language model with approximately **0.2 million parameters**, produced by knowledge distillation from [Supra-Mini-v4-2M](https://huggingface.co/SupraLabs/Supra-Mini-v4-2M).
25
+
26
+ It was trained 500 steps for 30 minutes on a GTX 750 Ti 4GB using generated text from the teacher.
27
+
28
+ ## Some outputs:
29
+
30
+ Prompt : Throughout history, great civilizations
31
+
32
+ Output: Throughout history, great civilizations to in, a be polrain for is with more the the be the for. of be of on (I.er The b M.A-R and or have that not is and the is this they, can for to to. is of a a, to ofs the for and the a. in the is to as of is that an that of and you the which is, the, for in be a are by’ of. and to a m
33
+
34
+ Prompt : The human brain is capable of
35
+ --------------------------------------------------
36
+ The human brain is capable ofs in an more that in a new can is the this the a of the pS, the a to the other in not it.
37
+ .. and with a to that be are of to for in of of ass. The be of the,.F-
38
+ s be the of dL
39
+ al. ins of be and of Sin: and or that a one that to and a a bFed, as
40
+ Ral., the, is a and as