turkish-gpt2-medium-finetuned-pdfs

This model is a fine-tuned version of ytu-ce-cosmos/turkish-gpt2-medium on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.6251

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 16
• total_train_batch_size: 64
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 1.0

Training results

Training Loss	Epoch	Step	Validation Loss
2.8161	0.0285	50	2.8298
2.7675	0.0569	100	2.7596
2.6885	0.0854	150	2.7324
2.6692	0.1139	200	2.7224
2.6849	0.1423	250	2.7088
2.6689	0.1708	300	2.7013
2.6558	0.1993	350	2.6972
2.6076	0.2277	400	2.6840
2.5762	0.2562	450	2.6823
2.6125	0.2847	500	2.6756
2.5573	0.3131	550	2.6679
2.6253	0.3416	600	2.6617
2.5285	0.3701	650	2.6608
2.523	0.3985	700	2.6525
2.4611	0.4270	750	2.6500
2.5456	0.4555	800	2.6462
2.5815	0.4840	850	2.6421
2.4772	0.5124	900	2.6398
2.5755	0.5409	950	2.6356
2.5165	0.5694	1000	2.6335
2.5441	0.5978	1050	2.6321
2.5212	0.6263	1100	2.6301
2.57	0.6548	1150	2.6283
2.5052	0.6832	1200	2.6277
2.5508	0.7117	1250	2.6271
2.4813	0.7402	1300	2.6261
2.5459	0.7686	1350	2.6257
2.4531	0.7971	1400	2.6255
2.4906	0.8256	1450	2.6253
2.5867	0.8540	1500	2.6251
2.5177	0.8825	1550	2.6251
2.4529	0.9110	1600	2.6251
2.5726	0.9394	1650	2.6251
2.5035	0.9679	1700	2.6251
2.5258	0.9964	1750	2.6251

Framework versions

• Transformers 4.55.2
• Pytorch 2.6.0+cu124
• Datasets 4.0.0
• Tokenizers 0.21.4