WajeehAzeemX/small-finetune-ar-az

Whisper Small Ar Tashkeel - AzeemX

This model is a fine-tuned version of arbml/whisper-small-ar on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0548
• Wer: 80.0678

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 7000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.1104	1.0320	1000	0.1108	95.5495
0.042	2.0640	2000	0.0791	60.0267
0.015	3.0960	3000	0.0622	61.8399
0.0051	4.1280	4000	0.0548	48.6347
0.0021	5.1600	5000	0.0521	67.4069
0.0007	6.1920	6000	0.0539	71.4779
0.0004	7.2239	7000	0.0548	80.0678

Framework versions

• Transformers 4.45.2
• Pytorch 2.3.1
• Datasets 3.0.1
• Tokenizers 0.20.0