fine-tuned-DatasetQAS-IDK-MRC-with-indobert-base-uncased-with-ITTL-without-freeze-LR-1e-05

This model is a fine-tuned version of indolem/indobert-base-uncased on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.0883
• Exact Match: 65.4450
• F1: 70.8022

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 32
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Exact Match	F1
6.2828	0.49	36	2.6576	49.7382	49.7756
3.794	0.98	72	1.9936	49.8691	49.8691
2.2086	1.47	108	1.8469	49.2147	49.5992
2.2086	1.96	144	1.7445	50.5236	51.9107
2.0123	2.46	180	1.6178	49.8691	54.4031
1.7802	2.95	216	1.4800	54.8429	58.8765
1.5945	3.44	252	1.3337	57.5916	62.8748
1.5945	3.93	288	1.3153	58.2461	63.4667
1.4083	4.42	324	1.2184	59.8168	65.4478
1.2513	4.91	360	1.2348	58.3770	64.1649
1.2513	5.4	396	1.1415	62.6963	68.0081
1.161	5.89	432	1.1463	62.6963	67.6633
1.0755	6.38	468	1.1126	63.4817	68.7554
1.0099	6.87	504	1.0823	63.4817	68.9182
1.0099	7.37	540	1.0547	66.2304	71.2423
0.9815	7.86	576	1.0835	63.4817	69.1031
0.9464	8.35	612	1.0644	66.3613	71.4374
0.9464	8.84	648	1.0642	65.9686	71.2813
0.9325	9.33	684	1.0786	65.4450	70.8541
0.913	9.82	720	1.0883	65.4450	70.8022

Framework versions

• Transformers 4.27.4
• Pytorch 1.13.1+cu117
• Datasets 2.2.0
• Tokenizers 0.13.2