card

README.md

@@ -15,7 +15,7 @@ metrics:
 
 ## Model description
 
-This model is a [RoBERTa base model](https://huggingface.co/roberta-base) pre-trained model further trained with masked language modeling task on a compendium of english scientific textual examples from the life sciences using the [BioLang dataset](https://huggingface.co/datasets/EMBO/biolang).
+This model is a [RoBERTa base pre-trained model](https://huggingface.co/roberta-base) that was further trained using a masked language modeling task on a compendium of english scientific textual examples from the life sciences using the [BioLang dataset](https://huggingface.co/datasets/EMBO/biolang).
 
 ## Intended uses & limitations
 
@@ -54,18 +54,18 @@ Training code is available at https://github.com/source-data/soda-roberta
 
 - Command: `python -m lm.train /data/json/oapmc_abstracts_figs/ MLM`
 - Tokenizer vocab size: 50265
-- Training data: bio_lang/MLM 
-- Training with: 12005390 examples.
-- Evaluating on: 36713 examples.
-- Epochs :3.0
-- per_device_train_batch_size: 16, 
-- per_device_eval_batch_size; 16, 
-- learning_rate: 5e-05, 
-- weight_decay: 0.0, 
-- adam_beta1: 0.9, 
-- adam_beta2: 0.999, 
-- adam_epsilon: 1e-08, 
-- max_grad_norm: 1.0, 
+- Training data: EMBO/biolang MLM
+- Training with: 12005390 examples
+- Evaluating on: 36713 examples
+- Epochs: 3.0
+- `per_device_train_batch_size`: 16
+- `per_device_eval_batch_size`: 16
+- `learning_rate`: 5e-05
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
 - tensorboard run: lm-MLM-2021-01-27T15-17-43.113766
 
 End of training: