idt5-base-qaqg-v1.72-TydiQA-id

This model is a fine-tuned version of muchad/idt5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.9052
• Rouge1: 0.5177
• Rouge2: 0.3448
• Rougel: 0.5147
• Rougelsum: 0.5150
• Bleu: 0.3292
• Rouge All: {'rouge1': 0.5177072322234892, 'rouge2': 0.34482574734600013, 'rougeL': 0.5147251357018411, 'rougeLsum': 0.5150166458813936}
• Bleu All: {'bleu': 0.32918730688074743, 'precisions': [0.5618240027688847, 0.4019407008086253, 0.31898907103825136, 0.27972027972027974], 'brevity_penalty': 0.8737306539256516, 'length_ratio': 0.8810703666997026, 'translation_length': 11557, 'reference_length': 13117}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Bleu	Rouge All	Bleu All
1.4155	1.0	2281	0.9862	0.4583	0.2938	0.4565	0.4567	0.2795	{'rouge1': 0.45831252200341155, 'rouge2': 0.29375171236948294, 'rougeL': 0.45647224292828814, 'rougeLsum': 0.4566568504580011}	{'bleu': 0.2795175311770661, 'precisions': [0.5381511027686532, 0.37155141526334645, 0.2961473000154727, 0.25978058373007656], 'brevity_penalty': 0.7936876940091591, 'length_ratio': 0.8123046428299153, 'translation_length': 10655, 'reference_length': 13117}
1.0779	2.0	4562	0.9434	0.4883	0.3156	0.4863	0.4868	0.3019	{'rouge1': 0.4883018577458882, 'rouge2': 0.31555198369418846, 'rougeL': 0.4862965635386509, 'rougeLsum': 0.48683782058576863}	{'bleu': 0.30194137259645165, 'precisions': [0.5481494789795185, 0.38429378531073444, 0.3034861854477072, 0.2653100403148397], 'brevity_penalty': 0.836679022056107, 'length_ratio': 0.848669665319814, 'translation_length': 11132, 'reference_length': 13117}
0.9104	3.0	6843	0.8954	0.5007	0.3299	0.4985	0.4984	0.3154	{'rouge1': 0.5006830238474206, 'rouge2': 0.32990889344944896, 'rougeL': 0.4985235066286749, 'rougeLsum': 0.4984076502044583}	{'bleu': 0.3153707762527571, 'precisions': [0.5588708236555298, 0.3986759425493716, 0.3170591614014934, 0.27838198817019655], 'brevity_penalty': 0.842157305881003, 'length_ratio': 0.8533963558740566, 'translation_length': 11194, 'reference_length': 13117}
0.7397	4.0	9124	0.9022	0.5115	0.3375	0.5092	0.5092	0.3272	{'rouge1': 0.511517333788021, 'rouge2': 0.33746425235616573, 'rougeL': 0.5091510133819286, 'rougeLsum': 0.5091748064951371}	{'bleu': 0.3271989795493179, 'precisions': [0.5580395528804815, 0.3971972614462987, 0.3132090761750405, 0.2753315649867374], 'brevity_penalty': 0.8799774600806517, 'length_ratio': 0.8866356636426012, 'translation_length': 11630, 'reference_length': 13117}
0.7146	5.0	11405	0.9052	0.5177	0.3448	0.5147	0.5150	0.3292	{'rouge1': 0.5177072322234892, 'rouge2': 0.34482574734600013, 'rougeL': 0.5147251357018411, 'rougeLsum': 0.5150166458813936}	{'bleu': 0.32918730688074743, 'precisions': [0.5618240027688847, 0.4019407008086253, 0.31898907103825136, 0.27972027972027974], 'brevity_penalty': 0.8737306539256516, 'length_ratio': 0.8810703666997026, 'translation_length': 11557, 'reference_length': 13117}

Framework versions

• Transformers 4.57.1
• Pytorch 2.9.0+cu129
• Datasets 4.2.0
• Tokenizers 0.22.1