xlsr-syntesized-turkish-4-hour-hlr

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1491
• Wer: 0.1312

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 2
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 200
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
6.3817	0.52	100	3.6970	1.0
3.1563	1.04	200	3.0338	1.0
2.0046	1.56	300	0.7680	0.6355
0.5563	2.08	400	0.3654	0.3463
0.3678	2.6	500	0.3055	0.3413
0.3173	3.12	600	0.2524	0.3021
0.2374	3.65	700	0.2030	0.2492
0.2194	4.17	800	0.1930	0.2411
0.1806	4.69	900	0.1653	0.1985
0.1603	5.21	1000	0.1562	0.2021
0.142	5.73	1100	0.1491	0.1820
0.1261	6.25	1200	0.1541	0.1787
0.1146	6.77	1300	0.1425	0.1784
0.1049	7.29	1400	0.1364	0.1803
0.095	7.81	1500	0.1331	0.1675
0.0887	8.33	1600	0.1349	0.1563
0.0772	8.85	1700	0.1330	0.1679
0.0732	9.38	1800	0.1354	0.1513
0.0735	9.9	1900	0.1390	0.1516
0.0701	10.42	2000	0.1530	0.1480
0.0642	10.94	2100	0.1428	0.1461
0.0611	11.46	2200	0.1366	0.1443
0.0584	11.98	2300	0.1485	0.1429
0.0587	12.5	2400	0.1554	0.1448
0.0559	13.02	2500	0.1430	0.1446
0.0502	13.54	2600	0.1448	0.1389
0.0519	14.06	2700	0.1397	0.1415
0.047	14.58	2800	0.1484	0.1403
0.0459	15.1	2900	0.1469	0.1437
0.0436	15.62	3000	0.1491	0.1340
0.039	16.15	3100	0.1521	0.1395
0.0406	16.67	3200	0.1513	0.1345
0.0391	17.19	3300	0.1483	0.1359
0.0376	17.71	3400	0.1487	0.1363
0.0383	18.23	3500	0.1478	0.1329
0.0394	18.75	3600	0.1486	0.1347
0.0361	19.27	3700	0.1480	0.1337
0.0342	19.79	3800	0.1491	0.1312

Framework versions

• Transformers 4.26.0
• Pytorch 2.0.1+cu118
• Datasets 2.9.0
• Tokenizers 0.13.3