fine-tuned-DatasetQAS-TYDI-QA-ID-with-indobert-base-uncased-with-ITTL-with-freeze-LR-1e-05

This model is a fine-tuned version of indolem/indobert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.3132
• Exact Match: 53.2628
• F1: 68.3641

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 16
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Exact Match	F1
6.3129	0.5	19	3.9006	5.6437	16.4748
6.3129	1.0	38	2.8272	17.1076	30.0839
3.8917	1.5	57	2.4681	18.8713	32.8962
3.8917	2.0	76	2.2891	25.3968	38.0874
3.8917	2.5	95	2.1835	26.9841	39.5053
2.3963	3.0	114	2.0885	28.5714	42.0243
2.3963	3.5	133	1.9971	32.4515	45.4085
2.112	4.0	152	1.9124	34.3915	48.2893
2.112	4.5	171	1.8358	37.0370	50.6492
2.112	5.0	190	1.7545	40.7407	54.7031
1.8205	5.5	209	1.6432	44.4444	58.2669
1.8205	6.0	228	1.5589	46.9136	60.8052
1.8205	6.5	247	1.4861	48.1481	62.5185
1.573	7.0	266	1.4381	49.7354	64.1985
1.573	7.5	285	1.3944	51.6755	66.0223
1.387	8.0	304	1.3534	53.2628	67.6841
1.387	8.5	323	1.3384	53.0864	67.8619
1.387	9.0	342	1.3344	52.9101	68.0618
1.2998	9.5	361	1.3182	53.2628	68.4149
1.2998	10.0	380	1.3132	53.2628	68.3641

Framework versions

• Transformers 4.26.1
• Pytorch 1.13.1+cu117
• Datasets 2.2.0
• Tokenizers 0.13.2