Eeppa commited on
Commit
59e2c62
·
verified ·
1 Parent(s): c360acf

Delete README.md

Browse files
Files changed (1) hide show
  1. README.md +0 -28
README.md DELETED
@@ -1,28 +0,0 @@
1
- # TinyBuddy-30M
2
-
3
- A 30 million parameter GPT-style transformer trained on TinyStories.
4
-
5
- ## Architecture
6
- - 6 layers, 8 attention heads, 256 embedding dim
7
- - 50,000 vocabulary size (untied weights)
8
- - 512 context length (trained on 128 for speed)
9
-
10
- ## Training
11
- - Dataset: TinyStories (5,000 stories)
12
- - Steps: 1,500
13
- - Hardware: CPU only
14
- - Loss: ~5.5 (coherent but not good)
15
-
16
- ## What It Can Do
17
- - Generate 2-3 word fragments that resemble story patterns
18
- - Sometimes repeat words from the prompt
19
- - Produce gibberish that's trying to be English
20
-
21
- ## What It Cannot Do
22
- - Tell a coherent story
23
- - Answer questions
24
- - Anything useful
25
-
26
- ## Why It Exists
27
- To demonstrate that even a tiny transformer learns *patterns*, not rules.
28
- This is a real AI, just a very small one.