MihaiPopa-1
/

CinnabarLM-4M-Base

Text Generation

Model card Files Files and versions

MihaiPopa-1 commited on 6 days ago

Commit

e925e08

·

verified ·

1 Parent(s): 85c4d58

Update README.md

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -32,7 +32,7 @@ Because it's a good idea to make tiny LLMs. Some people already did with [MicroL
 | Batch Size | 4 x 8 = 32 |
 | Context Window | Maybe 2048 tokens |
 | `hidden_size` | 192 |
-| `intermediate_size` | 192 |
 | `num_hidden_layers` | 6 |
 | `num_attention_heads` | 6 |
 | `max_position_embeddings` | 2048 |

 | Batch Size | 4 x 8 = 32 |
 | Context Window | Maybe 2048 tokens |
 | `hidden_size` | 192 |
+| `intermediate_size` | 512 |
 | `num_hidden_layers` | 6 |
 | `num_attention_heads` | 6 |
 | `max_position_embeddings` | 2048 |