idt5-base-qaqg-v1.12-TydiQA-id

This model is a fine-tuned version of muchad/idt5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.8697
• Rouge1: 0.5240
• Rouge2: 0.3498
• Rougel: 0.5227
• Rougelsum: 0.5223
• Bleu: 0.3432
• Rouge All: {'rouge1': 0.5240346007536814, 'rouge2': 0.34978996264301754, 'rougeL': 0.522710963680047, 'rougeLsum': 0.5223208797539063}
• Bleu All: {'bleu': 0.34315353770773166, 'precisions': [0.5807064513343121, 0.4227170054856405, 0.3432937644972029, 0.3066427289048474], 'brevity_penalty': 0.855878745494461, 'length_ratio': 0.8653314401016367, 'translation_length': 11579, 'reference_length': 13381}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Bleu	Rouge All	Bleu All
1.4087	1.0	2281	0.9875	0.4623	0.2939	0.4613	0.4606	0.3128	{'rouge1': 0.46229438264513656, 'rouge2': 0.2938529297746283, 'rougeL': 0.4612646163282077, 'rougeLsum': 0.4605582862662171}	{'bleu': 0.3127745735621603, 'precisions': [0.5066238458450422, 0.352172203656379, 0.2824858757062147, 0.25564860167483017], 'brevity_penalty': 0.9283488958054709, 'length_ratio': 0.9307973992975114, 'translation_length': 12455, 'reference_length': 13381}
1.0619	2.0	4562	0.8708	0.5134	0.3414	0.5112	0.5108	0.3338	{'rouge1': 0.513394588591981, 'rouge2': 0.34136043067199673, 'rougeL': 0.5112324089779485, 'rougeLsum': 0.5108241780391356}	{'bleu': 0.33378331860538335, 'precisions': [0.56817986784519, 0.4075338811226123, 0.3296777687516924, 0.29425369151396547], 'brevity_penalty': 0.8621827752794875, 'length_ratio': 0.8708616695314252, 'translation_length': 11653, 'reference_length': 13381}
0.8994	3.0	6843	0.8689	0.5195	0.3484	0.5181	0.5180	0.3432	{'rouge1': 0.5194791000460289, 'rouge2': 0.34835227420426074, 'rougeL': 0.5181144219443232, 'rougeLsum': 0.5179952388777813}	{'bleu': 0.34316067855090626, 'precisions': [0.5818499127399651, 0.4252560470690782, 0.3474458634092171, 0.31538320161734973], 'brevity_penalty': 0.8456696104377565, 'length_ratio': 0.8564382333158956, 'translation_length': 11460, 'reference_length': 13381}
0.7231	4.0	9124	0.8536	0.5132	0.3361	0.5122	0.5117	0.3347	{'rouge1': 0.5131997890831617, 'rouge2': 0.33609967105633076, 'rougeL': 0.512183530122801, 'rougeLsum': 0.5117135378316945}	{'bleu': 0.33466149600472417, 'precisions': [0.579715271023079, 0.42084856541486654, 0.3420865862313698, 0.31277117524995285], 'brevity_penalty': 0.8325875421766096, 'length_ratio': 0.8451535759659218, 'translation_length': 11309, 'reference_length': 13381}
0.6618	5.0	11405	0.8697	0.5240	0.3498	0.5227	0.5223	0.3432	{'rouge1': 0.5240346007536814, 'rouge2': 0.34978996264301754, 'rougeL': 0.522710963680047, 'rougeLsum': 0.5223208797539063}	{'bleu': 0.34315353770773166, 'precisions': [0.5807064513343121, 0.4227170054856405, 0.3432937644972029, 0.3066427289048474], 'brevity_penalty': 0.855878745494461, 'length_ratio': 0.8653314401016367, 'translation_length': 11579, 'reference_length': 13381}

Framework versions

• Transformers 4.57.1
• Pytorch 2.9.0+cu129
• Datasets 4.2.0
• Tokenizers 0.22.1