idt5-base-qaqg-v2.42-TydiQA-id

This model is a fine-tuned version of muchad/idt5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.3146
• Rouge1: 0.5205
• Rouge2: 0.3388
• Rougel: 0.5195
• Rougelsum: 0.5189
• Bleu: 0.3306
• Rouge All: {'rouge1': 0.520475321144294, 'rouge2': 0.33882226793500336, 'rougeL': 0.5195448292360988, 'rougeLsum': 0.5189371840267327}
• Bleu All: {'bleu': 0.33058685697627865, 'precisions': [0.5704316360816748, 0.41126005361930296, 0.33139219934994585, 0.29467915856461024], 'brevity_penalty': 0.8497332746028831, 'length_ratio': 0.859968881973772, 'translation_length': 11607, 'reference_length': 13497}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Bleu	Rouge All	Bleu All
1.7678	1.0	2281	1.4692	0.4713	0.2989	0.4698	0.4702	0.2970	{'rouge1': 0.4713266683951046, 'rouge2': 0.2989045132797604, 'rougeL': 0.4697835172108892, 'rougeLsum': 0.4702093338113894}	{'bleu': 0.29702792776916115, 'precisions': [0.5225048923679061, 0.3626860943934115, 0.28946666666666665, 0.25688873386815486], 'brevity_penalty': 0.8620968620123841, 'length_ratio': 0.8707861006149514, 'translation_length': 11753, 'reference_length': 13497}
1.3412	2.0	4562	1.3335	0.5016	0.3286	0.5011	0.5008	0.3212	{'rouge1': 0.5015613716925748, 'rouge2': 0.32862953603969414, 'rougeL': 0.501113409343755, 'rougeLsum': 0.5008176867820567}	{'bleu': 0.32118915563457645, 'precisions': [0.5431164901664145, 0.38737520798668884, 0.31189415771548334, 0.2776354006492397], 'brevity_penalty': 0.8742469997474056, 'length_ratio': 0.8815292287174927, 'translation_length': 11898, 'reference_length': 13497}
1.1348	3.0	6843	1.3020	0.5159	0.3374	0.5148	0.5144	0.3280	{'rouge1': 0.5158865657028175, 'rouge2': 0.33736768096786773, 'rougeL': 0.5147557074306568, 'rougeLsum': 0.5143776441393975}	{'bleu': 0.3279568905681555, 'precisions': [0.5500912257422458, 0.3926963993453355, 0.3141394753678823, 0.2747688243064729], 'brevity_penalty': 0.8875061240164817, 'length_ratio': 0.8933837148996073, 'translation_length': 12058, 'reference_length': 13497}
1.0722	4.0	9124	1.2946	0.5155	0.3382	0.5145	0.5142	0.3283	{'rouge1': 0.5155167992697491, 'rouge2': 0.33824736223723784, 'rougeL': 0.5145299267771566, 'rougeLsum': 0.5142343275940109}	{'bleu': 0.32828344879392773, 'precisions': [0.5645009012101966, 0.40655352759099156, 0.32821548821548824, 0.2905997895475272], 'brevity_penalty': 0.8534730308844034, 'length_ratio': 0.8632288656738535, 'translation_length': 11651, 'reference_length': 13497}
0.926	5.0	11405	1.3146	0.5205	0.3388	0.5195	0.5189	0.3306	{'rouge1': 0.520475321144294, 'rouge2': 0.33882226793500336, 'rougeL': 0.5195448292360988, 'rougeLsum': 0.5189371840267327}	{'bleu': 0.33058685697627865, 'precisions': [0.5704316360816748, 0.41126005361930296, 0.33139219934994585, 0.29467915856461024], 'brevity_penalty': 0.8497332746028831, 'length_ratio': 0.859968881973772, 'translation_length': 11607, 'reference_length': 13497}

Framework versions

• Transformers 4.57.1
• Pytorch 2.9.0+cu129
• Datasets 4.2.0
• Tokenizers 0.22.1