Update README.md

README.md

@@ -19,7 +19,7 @@ Jam-CGPT is a GPT2-like model that follows [jam](https://huggingface.co/apcl/jam
 |a | accumulation steps				| 2 |		 
 |d | dropout							| 0.20 |		 
 |r | learning rate                     | 3e-5 |		 
-|y | weight decay						| 1e-5 |	
+|y | iterations						| 1e-5 |	
 |iter | number of iterations after pretraing						| 757,000 |	
 
 ## Jam-CGPT 110 million parameters model
@@ -33,7 +33,7 @@ Jam-CGPT is a GPT2-like model that follows [jam](https://huggingface.co/apcl/jam
 |a | accumulation steps				| 4 |		 
 |d | dropout							| 0.20 |		 
 |r | learning rate                     | 3e-5 |		 
-|y | weight decay						| 1e-5 |	
+|y | iterations						| 1e-5 |	
 |iter | number of iterations after pretraing						| 762,000 |	
 
 
@@ -49,7 +49,7 @@ Jam-CGPT is a GPT2-like model that follows [jam](https://huggingface.co/apcl/jam
 |d | dropout							| 0.20 |		 
 |r | learning rate                     | 3e-5 |		 
 |y | weight decay						| 1e-5 |
-|iter | weight decay						| 272,000 |
+|iter | iterations					| 272,000 |
 
 - Note that you can adjust the batch size and accumulation steps based on your GPU memory. But, the batch size * accumulation steps should be 128.
 - If you finetune your models with multiple GPUs, you can turn down accumulation steps. For example, if you finetune with 2 GPUs, you will need to half the accumulation steps.