Update README.md

README.md

@@ -24,7 +24,20 @@ tags:
 
 **DistillSupra-0.2M** is an ultra-compact causal language model with approximately **0.2 million parameters**, produced by knowledge distillation from [Supra-Mini-v4-2M](https://huggingface.co/SupraLabs/Supra-Mini-v4-2M).
 
-It was trained 500 steps for 30 minutes on a GTX 750 Ti 4GB using generated text from the teacher.
+It was trained 500 steps(1 Epoch) for 30 minutes on a GTX 750 Ti 4GB using generated text from the teacher.
+
+The model was **10x** compressed! That's crazy!
+
+## Architecture
+
+| Parameter          | Teacher | Student |
+|---------------------|---------|---------|
+| hidden_size         | 64      | 48      |
+| intermediate_size   | 128     | 96      |
+| num_hidden_layers   | 5       | 4       |
+| num_attention_heads | 8       | 6       |
+| vocab_size          | 4096    | 4096    |
+| Parameters         | ~468k   | ~289k   |
 
 ## Some outputs:
 
@@ -38,4 +51,12 @@ Output: The human brain is capable ofs in an more that in a new can is the this
 
 Prompt : The most important principle in science is
 --------------------------------------------------
-The most important principle in science is a is a this are not for that the to of be digels-LC. to the in a the to, on to,
+The most important principle in science is a is a this are not for that the to of be digels-LC. to the in a the to, on to,
+
+## Why did supra created this trash?
+
+We are currently researching knowledge distillation and this was the first step! Things will better up!
+
+## Final Thought
+
+Knowledge distillation is a promising thing for us, we believe that LLMs can be helpful even being so small!