Commit ·
12e3bd8
1
Parent(s): b1a94f5
Update README.md
Browse files
README.md
CHANGED
|
@@ -4,13 +4,11 @@ license: apache-2.0
|
|
| 4 |
|
| 5 |
Contains file for Transformer model that answers 5-digit addition questions (e.g. 12345+67890=) with near zero low loss.
|
| 6 |
Model has answered 1 million addition questions with any errors.
|
| 7 |
-
Model has 2 layers, 3 attention heads, d-model = 510, d-head = 170, and was trained for 30K epochs.
|
| 8 |
|
|
|
|
| 9 |
The CoLab used to train the model is here:
|
| 10 |
https://github.com/PhilipQuirke/transformer-maths/blob/main/assets/Accurate_Addition_Train.ipynb
|
| 11 |
|
| 12 |
The CoLab used to analyse the model is here:
|
| 13 |
https://github.com/PhilipQuirke/transformer-maths/blob/main/assets/Accurate_Addition_Analyse.ipynb
|
| 14 |
|
| 15 |
-
|
| 16 |
-
|
|
|
|
| 4 |
|
| 5 |
Contains file for Transformer model that answers 5-digit addition questions (e.g. 12345+67890=) with near zero low loss.
|
| 6 |
Model has answered 1 million addition questions with any errors.
|
|
|
|
| 7 |
|
| 8 |
+
Model has 2 layers, 3 attention heads, d-model = 510, d-head = 170, and was trained for 30K epochs.
|
| 9 |
The CoLab used to train the model is here:
|
| 10 |
https://github.com/PhilipQuirke/transformer-maths/blob/main/assets/Accurate_Addition_Train.ipynb
|
| 11 |
|
| 12 |
The CoLab used to analyse the model is here:
|
| 13 |
https://github.com/PhilipQuirke/transformer-maths/blob/main/assets/Accurate_Addition_Analyse.ipynb
|
| 14 |
|
|
|
|
|
|