idt5-base-qaqg-v2.72-SQuAD-id

This model is a fine-tuned version of muchad/idt5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.4085
• Rouge1: 0.4109
• Rouge2: 0.2278
• Rougel: 0.3904
• Rougelsum: 0.3905
• Bleu: 0.1811
• Rouge All: {'rouge1': 0.41087088026034857, 'rouge2': 0.2277534250499209, 'rougeL': 0.3904240277727141, 'rougeLsum': 0.390494951725938}
• Bleu All: {'bleu': 0.18109175028571028, 'precisions': [0.4617268453567649, 0.2618181293135811, 0.1697900280418576, 0.11328468346926678], 'brevity_penalty': 0.8246726889770346, 'length_ratio': 0.8383855058707002, 'translation_length': 81258, 'reference_length': 96922}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Bleu	Rouge All	Bleu All
1.7486	1.0	12000	1.4755	0.3778	0.2007	0.3586	0.3589	0.1508	{'rouge1': 0.3777817375364356, 'rouge2': 0.20065247649722928, 'rougeL': 0.3586063469996108, 'rougeLsum': 0.3588968179939111}	{'bleu': 0.15084689869993542, 'precisions': [0.4445770178843027, 0.23727204829775317, 0.14988417276939173, 0.09674715716033162], 'brevity_penalty': 0.7627627782530312, 'length_ratio': 0.7869008068343617, 'translation_length': 76268, 'reference_length': 96922}
1.5034	2.0	24000	1.4297	0.3942	0.2145	0.3728	0.3730	0.1834	{'rouge1': 0.3942331055347321, 'rouge2': 0.2145047559038011, 'rougeL': 0.3728192692593151, 'rougeLsum': 0.37296589472381403}	{'bleu': 0.18340626423986411, 'precisions': [0.42978353217642806, 0.24049822250104558, 0.15605768347570498, 0.10258136108129741], 'brevity_penalty': 0.9093589619256438, 'length_ratio': 0.9132291946100989, 'translation_length': 88512, 'reference_length': 96922}
1.3516	3.0	36000	1.4047	0.3984	0.2165	0.3784	0.3786	0.1674	{'rouge1': 0.3984459968893157, 'rouge2': 0.2164753531291516, 'rougeL': 0.3783699991601624, 'rougeLsum': 0.37858365281329065}	{'bleu': 0.16738698968772053, 'precisions': [0.45486509306751016, 0.25006310225534883, 0.15943612334801763, 0.1049634020292362], 'brevity_penalty': 0.8013689050085329, 'length_ratio': 0.818709890427354, 'translation_length': 79351, 'reference_length': 96922}
1.197	4.0	48000	1.4074	0.4056	0.2238	0.3855	0.3856	0.1690	{'rouge1': 0.40562656793936586, 'rouge2': 0.2238097169341151, 'rougeL': 0.38548337863789967, 'rougeLsum': 0.3855655064907162}	{'bleu': 0.1690160104884448, 'precisions': [0.46631290168854084, 0.2594415075943389, 0.165731612106687, 0.10989848821504815], 'brevity_penalty': 0.7800983427723612, 'length_ratio': 0.8010668372505726, 'translation_length': 77641, 'reference_length': 96922}
1.1207	5.0	60000	1.4085	0.4109	0.2278	0.3904	0.3905	0.1811	{'rouge1': 0.41087088026034857, 'rouge2': 0.2277534250499209, 'rougeL': 0.3904240277727141, 'rougeLsum': 0.390494951725938}	{'bleu': 0.18109175028571028, 'precisions': [0.4617268453567649, 0.2618181293135811, 0.1697900280418576, 0.11328468346926678], 'brevity_penalty': 0.8246726889770346, 'length_ratio': 0.8383855058707002, 'translation_length': 81258, 'reference_length': 96922}

Framework versions

• Transformers 4.57.1
• Pytorch 2.9.0+cu129
• Datasets 4.2.0
• Tokenizers 0.22.1