Harley-ml commited on
Commit
3436ea4
·
verified ·
1 Parent(s): 55ff9eb

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -31,7 +31,7 @@ Author: Paul Courneya (Harley-ml)
31
  ## **Description**
32
 
33
  Dillion is a 1.2M parameter language model trained on ~9B tokens of FineWeb-edu.
34
- Our goal was to make one of the best sub-1.5M parameter LMs through depth (12 layers) and huge overtraining (~8900 tokens per parameter).
35
  Dillion beats or ties with models much larger than itself such as SupraMini-v4-2M and Tenete-8M.
36
 
37
  ## Architecture
 
31
  ## **Description**
32
 
33
  Dillion is a 1.2M parameter language model trained on ~9B tokens of FineWeb-edu.
34
+ Our goal was to make one of the best sub-1.5M parameter LMs through depth (12 layers) and huge overtraining (about 8900 tokens per parameter).
35
  Dillion beats or ties with models much larger than itself such as SupraMini-v4-2M and Tenete-8M.
36
 
37
  ## Architecture