Update README.md

README.md

@@ -21,41 +21,54 @@ library_name: transformers
 ---
 ## **i'm not releasing yet LH, if you want(it probably need it) changes, you can change!!!**!
 
-## **MicroSupra-1k**
+## **🦅 MicroSupra-1k**
 
 So... have you ever seen a model that runs on a 3 dollars hardware? No? If no, Now you're seeing!
 
-MicroSupra-1k is a 1k parameters model trained on 300M Tokens of FineWeb-Edu for 3 epochs for 1 minute(Yes! 1 minute!) on a GTX 750Ti 4GB(AxionLab Hardware)
+MicroSupra-1k is an bacteria base model(lol) trained on 300 million tokens of Fineweb-Edu for 3 epochs as the **first version** of our MicroSupra.
 
-**Some model outputs:**
+## Model Config
 
-[*] Prompt: The main concept of physics is
+- Parameters: 1046 (0.001M)
+- Architecture: LLaMa
+- Vocab size: 1024
+- Hidden Size: 1
+- Intermediate Size: 2
+- Hidden Layers: 1
+- Attention Heads: 1
+- Max Position Embeddings: 256
+- Learning rate: <code>5e-3<code>
 
-[*] Output: The main concept of physics is  a,s and the. thet to, theing.... the,a then,c,i to, thee in b. toed.,,e theyalp the in,er thees- s,el,,,,
- and, the of ine,,s the of cs of thesss the. f. to. thesining andor dar,,al the,. of p.
- the.s the.,,s. anded,e. of, ofed, l toinging and themsr the of of. to
- to thes thes aen,., ofes of a.
+## Final Loss
+This model reached a final train loss after 3 epochs of **6.046**.
 
-[*] Prompt: My name is
-
-[*] Output: My name is edie and. as the, to. the, iningt thee the ofingi in
+## Examples
+**Prompt:** "My name is "<br>
+**Output:**: *"My name is ed and. as the, to. the, in
+ingt thee the ofingi in
  the., anda.-eo
  ofles, b the,er,s fing.ssp the the
 , of of, the,al, d to the m, the, to toed,
 seng,,.y. in the,., in and them the thened.sing to
  the of of andan the the,, the
- to..,,sing,,.aring the the. of.al.,s ofcal ar s..e and.sssor of, and and.
-
-[*] Prompt: Question: What is the capital of France?\nAnswer:
-
-[*] Output: Question: What is the capital of France?
-
-Answer:,. and to the. toc. ofs the m,a thee.. the, f ofling. as.,,y bt, the p, in, the,,ees toed ing to.o,
+ to..,,sing,,.aring the the. of.al.,s ofcal ar s..e and.sssor of, and and."*
+<br><br>
+**Prompt:** "The main concept of physics is "<br>
+**Output:**: *"The main concept of physics is  a,
+s and the. thet to, theing.... the,a then,c,i to, thee in b. toed.,,e theyalp the in,er thees- s,el,,,,
+ and, the of ine,,s the of cs of thesss the. f. to. thesining andor dar,,al the,. of p.
+ the.s the.,,s. anded,e. of, ofed, l toinging and themsr the of of. to
+ to thes thes aen,., ofes of a."*
+<br><br>
+**Prompt:** "Question: What is the capital of France?\nAnswer: "<br>
+**Output:**: *"Question: What is the capital of France?
+Answer:,. and to the. toc. ofs the m,a thee.. the, f ofling. as.,,y bt, the p
+, in, the,,ees toed ing to.
+o,
  thes. the..,s the.ed and andang,,ed the of,,ms. of, thei the, the,ey,,s l.ing toe the the,se the to, the, the,aror, the of-. in the. the. the,e the of ds to,ic the the aal at the..
-ingssy s and and
-
+ingssy s and and"*
 
-## Get Started 🚀
+## Usage 🚀
 
 ```python3
 print("[*] Loading libraries...")
@@ -71,7 +84,7 @@ print("[*] Loading model...")
 model = LlamaForCausalLM.from_pretrained(model_path)
 model.eval()
 
-prompt = "The most intelligent person on the world is "
+prompt = "Question: What is the capital of France?\nAnswer:"
 print(f"[*] Prompt: {prompt!r}")
 
 inputs = tokenizer(prompt, return_tensors="pt")
@@ -90,20 +103,16 @@ with torch.no_grad():
     )
 
 print("[*] Output:", tokenizer.decode(outputs[0], skip_special_tokens=True))
-```
 
-
-**🚫What the model CAN'T do:**
-Think
-Chat
-even predict the next token correctly lol
-
-**Why SupraLabs created this???**
+```
+## Why SupraLabs created this???
 
 Because we are experimenting sizes, experiments(like 1Bit quant, distillation(NEW THINGS ARE COMING WITH DISTILLATION! GET TUNED!), pruning) all to better your experience! We are working to big things!
 
-**Final thought**
+## Training guide
+We trained MicroSupra on a GTX750 Ti 4GB in 1 Minute for 3 epochs.<br>
+The model was trained on the first 300 million tokens of Sample-10BT from Fineweb-Edu using streaming tokenization.
 
-Even without any intelligence, it shows that scaling laws are real. This ant model doesn't know how to talk, but we all know it emotions 🤖🫶
+## Final thoughts
 
-SupraLabs are excited to work at more things for you!
+Even without any intelligence, it shows that scaling laws are real. This ant model doesn't know how to talk, but we all know it emotions 🤖🫶