AxionLab-official commited on
Commit
49c5dcd
·
verified ·
1 Parent(s): 015e1c1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +48 -2
README.md CHANGED
@@ -12,12 +12,58 @@ tags:
12
  - supra
13
  - SupraLabs
14
  - gtx
15
- - gpu
16
  - rtx
17
  - nvidia
18
  - llama
 
 
 
19
  ---
20
 
21
  ## **MicroSupra-1k**
22
 
23
- This model is being finished yet LH Tech, but here are the weights
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
12
  - supra
13
  - SupraLabs
14
  - gtx
 
15
  - rtx
16
  - nvidia
17
  - llama
18
+ - lhtech
19
+ - axionlab
20
+ library_name: transformers
21
  ---
22
 
23
  ## **MicroSupra-1k**
24
 
25
+ So... have you ever seen a model that runs on a 3 dollars hardware? No? If no, Now you're seeing!
26
+
27
+ MicroSupra-1k is a 1k parameters model trained on 300M Tokens of FineWeb-Edu for 1 minute(Yes! 59 seconds!) on a GTX 750Ti 4GB(AxionLab Hardware)
28
+
29
+ **Some model outputs:**
30
+
31
+ [*] Prompt: The main concept of physics is
32
+
33
+ [*] Output: The main concept of physics is a,s and the. thet to, theing.... the,a then,c,i to, thee in b. toed.,,e theyalp the in,er thees- s,el,,,,
34
+ and, the of ine,,s the of cs of thesss the. f. to. thesining andor dar,,al the,. of p.
35
+ the.s the.,,s. anded,e. of, ofed, l toinging and themsr the of of. to
36
+ to thes thes aen,., ofes of a.
37
+
38
+ [*] Prompt: My name is
39
+
40
+ [*] Output: My name is ed and. as the, to. the, iningt thee the ofingi in
41
+ the., anda.-eo
42
+ ofles, b the,er,s fing.ssp the the
43
+ , of of, the,al, d to the m, the, to toed,
44
+ seng,,.y. in the,., in and them the thened.sing to
45
+ the of of andan the the,, the
46
+ to..,,sing,,.aring the the. of.al.,s ofcal ar s..e and.sssor of, and and.
47
+
48
+ [*] Prompt: Question: What is the capital of France?\nAnswer:
49
+
50
+ [*] Output: Question: What is the capital of France?
51
+
52
+ Answer:,. and to the. toc. ofs the m,a thee.. the, f ofling. as.,,y bt, the p, in, the,,ees toed ing to.o,
53
+ thes. the..,s the.ed and andang,,ed the of,,ms. of, thei the, the,ey,,s l.ing toe the the,se the to, the, the,aror, the of-. in the. the. the,e the of ds to,ic the the aal at the..
54
+ ingssy s and and
55
+
56
+ **🚫What the model CAN'T do:**
57
+ Think
58
+ Chat
59
+ even predict the next token correctly lol
60
+
61
+ **Why SupraLabs created this???**
62
+
63
+ Because we are experimenting sizes, experiments(like 1Bit quant, distillation(NEW THINGS ARE COMING WITH DISTILLATION! GET TUNED!), pruning, all to better your experience! We are working to big things!)
64
+
65
+ **Final thought**
66
+
67
+ Even without any intelligence, it shows that scaling laws are real. This ant model doesn't know how to talk, but we all know it emotions 🤖🫶
68
+
69
+ SupraLabs are excited to work at more things to you!