abs-bf16-vl-7b-multimodal-reasoning-147k-16-train-lr2e-5-g0.997-l1.0-gpu4-bs8-ga32-ep2-wu50-cut4000

This model is a fine-tuned version of Qwen/Qwen2.5-VL-3B-Instruct on the vl_7b_multimodal_reasoning_147k_16_train dataset. It achieves the following results on the evaluation set:

Loss: 0.0030
Token Mean Mae: 93937469.0216
Token Mean Rmse: 128139.5714
Token Mean Seq Mean Mae: 11118.2995
Token Mean Seq Mean Rmse: 674.6324
Token Mean Relerr: 0.2571
Token Mean Seq Mean Relerr: 0.2767

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 8
eval_batch_size: 8
seed: 0
distributed_type: multi-GPU
num_devices: 4
gradient_accumulation_steps: 32
total_train_batch_size: 1024
total_eval_batch_size: 32
optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: cosine
lr_scheduler_warmup_steps: 50
num_epochs: 2.0

Training results

Training Loss	Epoch	Step	Validation Loss	Token Mean Mae	Token Mean Rmse	Token Mean Seq Mean Mae	Token Mean Seq Mean Rmse	Token Mean Relerr	Token Mean Seq Mean Relerr
0.0065	0.0218	50	0.0053	128590372.1979	168554.3870	15230.6997	929.8801	0.4148	0.5591
0.0038	0.0437	100	0.0040	112643657.9118	151358.8673	13332.7312	797.3027	0.3347	0.4112
0.0039	0.0655	150	0.0038	107764473.4941	146994.7216	12740.7590	768.1790	0.2927	0.3358
0.0041	0.0874	200	0.0038	107668764.4080	143977.2960	12736.8411	775.6873	0.3307	0.3831
0.0036	0.1092	250	0.0036	103651189.3526	137524.9949	12272.4688	744.6927	0.2819	0.3182
0.0035	0.1311	300	0.0036	104407289.0385	141889.1586	12352.7970	741.2913	0.3010	0.3475
0.0038	0.1529	350	0.0035	103700838.6377	139759.6069	12279.9324	742.6524	0.3067	0.3493
0.0032	0.1748	400	0.0036	103459928.6825	134688.1577	12259.3960	750.9565	0.3085	0.3463
0.0034	0.1966	450	0.0036	104623651.3055	136415.9298	12401.6634	751.1772	0.3222	0.3654
0.0032	0.2185	500	0.0035	102672724.7862	133724.2418	12171.8804	736.8338	0.3044	0.3303
0.0032	0.2403	550	0.0034	100910791.7469	139323.6622	11912.4688	718.4168	0.2719	0.2920
0.0035	0.2622	600	0.0034	101376518.0658	137700.2844	11996.3797	725.3526	0.2770	0.2999
0.0037	0.2840	650	0.0034	101816907.1862	141143.3144	12032.9882	718.2669	0.2713	0.2899
0.0031	0.3059	700	0.0036	105109547.2483	147335.8579	12401.1019	734.3813	0.2835	0.3040
0.0034	0.3277	750	0.0034	100639039.5566	142749.3821	11848.0665	714.1509	0.2757	0.2986
0.0035	0.3496	800	0.0034	102147814.3178	141729.6175	12074.3046	720.8283	0.2919	0.3328
0.0032	0.3714	850	0.0033	99722083.5412	135515.3395	11778.5753	713.2526	0.2910	0.3195
0.0032	0.3932	900	0.0033	99518793.3427	135422.7542	11765.9997	708.2480	0.2698	0.2885
0.0030	0.4151	950	0.0033	99424697.6673	136159.3559	11736.4889	709.4589	0.2797	0.3043
0.0034	0.4369	1000	0.0033	99879528.6343	139436.7020	11789.2454	706.3349	0.2702	0.2955
0.0031	0.4588	1050	0.0033	98859117.8559	133089.6709	11690.2475	707.8783	0.2961	0.3252
0.0030	0.4806	1100	0.0032	99464848.7481	135928.6823	11759.8965	707.9527	0.2864	0.3113
0.0038	0.5025	1150	0.0034	101214215.5811	134217.0886	11971.4681	730.9397	0.3145	0.3585
0.0033	0.5243	1200	0.0032	99185515.0407	138553.4722	11713.0062	707.6130	0.2751	0.3004
0.0033	0.5462	1250	0.0033	99847798.0965	137163.4793	11780.8793	711.2098	0.2948	0.3283
0.0030	0.5680	1300	0.0032	98180293.1573	134328.9631	11613.0891	698.5921	0.2683	0.2856
0.0032	0.5899	1350	0.0032	98206538.3301	135710.0979	11590.2239	701.3882	0.2735	0.2996
0.0035	0.6117	1400	0.0032	97984593.8830	134876.8108	11560.0282	699.7256	0.2686	0.2875
0.0033	0.6336	1450	0.0032	98360783.8113	138226.4750	11597.4974	698.5031	0.2636	0.2861
0.0034	0.6554	1500	0.0033	100186479.3599	141040.4543	11823.9163	703.8644	0.2649	0.2843
0.0030	0.6773	1550	0.0032	98015058.4708	135224.7889	11570.1882	697.7360	0.2802	0.3074
0.0030	0.6991	1600	0.0032	98208850.5701	137216.5164	11566.7801	698.5327	0.2745	0.2993
0.0030	0.7209	1650	0.0032	97767096.3238	133698.9627	11550.7590	696.7753	0.2820	0.3103
0.0028	0.7428	1700	0.0031	97131312.1439	131975.1368	11485.5284	693.6183	0.2703	0.2928
0.0032	0.7646	1750	0.0032	97637636.4863	129697.5189	11551.8404	706.1598	0.2773	0.3016
0.0034	0.7865	1800	0.0032	97266395.9038	133952.1093	11481.7876	696.4654	0.2731	0.2945
0.0031	0.8083	1850	0.0032	96926308.6240	131325.3286	11467.1931	692.3476	0.2688	0.2868
0.0031	0.8302	1900	0.0032	97153830.5935	133120.9201	11469.9680	693.8857	0.2745	0.2980
0.0030	0.8520	1950	0.0032	96935039.2931	132305.5800	11446.9276	694.6933	0.2657	0.2882
0.0032	0.8739	2000	0.0031	96968937.1622	135023.2929	11432.5382	691.1709	0.2734	0.2964
0.0029	0.8957	2050	0.0032	97453927.3902	129393.1500	11527.3802	700.9575	0.2965	0.3321
0.0030	0.9176	2100	0.0031	97039851.9726	135435.4182	11429.3101	691.8383	0.2726	0.3024
0.0026	0.9394	2150	0.0031	96191839.8209	131307.9334	11359.8603	685.9598	0.2554	0.2726
0.0031	0.9613	2200	0.0031	95455682.4813	134230.9557	11235.5411	681.9591	0.2611	0.2799
0.0030	0.9831	2250	0.0031	96009314.5315	133879.1594	11307.0007	685.4788	0.2612	0.2809
0.0031	1.0048	2300	0.0031	96620911.9819	133067.2467	11404.3217	690.7271	0.2746	0.3034
0.0024	1.0267	2350	0.0032	95741785.0033	131618.3764	11295.4543	686.5553	0.2517	0.2698
0.0025	1.0485	2400	0.0031	96333779.5558	136052.4330	11315.2061	684.5834	0.2647	0.2865
0.0027	1.0703	2450	0.0031	96119786.3267	126161.3694	11409.6150	695.8273	0.2787	0.3077
0.0029	1.0922	2500	0.0031	95276833.9024	128075.9928	11276.8539	684.0838	0.2638	0.2884
0.0027	1.1140	2550	0.0031	95447486.6730	127146.5506	11302.4921	685.9035	0.2681	0.2938
0.0027	1.1359	2600	0.0031	95961496.0961	130812.6475	11360.8824	687.3670	0.2562	0.2736
0.0028	1.1577	2650	0.0031	96267055.9699	132996.8830	11388.1030	685.3716	0.2555	0.2725
0.0028	1.1796	2700	0.0032	95903933.2110	130867.2700	11331.9102	685.3870	0.2520	0.2681
0.0028	1.2014	2750	0.0031	95778088.4914	133812.2100	11291.9611	684.2959	0.2541	0.2677
0.0026	1.2233	2800	0.0031	95345246.5155	129676.7091	11270.8655	685.2084	0.2664	0.2888
0.0026	1.2451	2850	0.0031	95432562.0290	130315.6608	11297.7553	684.8648	0.2568	0.2739
0.0028	1.2670	2900	0.0031	96158968.1498	134439.4791	11356.2583	683.8818	0.2583	0.2736
0.0026	1.2888	2950	0.0031	96145751.3856	133815.2061	11360.0251	683.2588	0.2508	0.2648
0.0026	1.3107	3000	0.0031	95561813.9464	130790.9639	11306.1160	682.5582	0.2509	0.2648
0.0028	1.3325	3050	0.0031	95858149.5401	131979.7433	11337.1636	684.0860	0.2685	0.2908
0.0028	1.3544	3100	0.0031	94942430.4530	127382.4337	11251.1711	682.7818	0.2623	0.2789
0.0035	1.3762	3150	0.0031	94933692.5256	129358.5440	11236.4957	681.6743	0.2640	0.2826
0.0026	1.3981	3200	0.0031	94713972.3401	127621.0320	11217.3753	681.5312	0.2593	0.2782
0.0027	1.4199	3250	0.0031	94848569.3951	127942.8143	11246.0819	682.2818	0.2572	0.2758
0.0028	1.4417	3300	0.0031	94138397.1890	125210.7266	11184.4835	678.8851	0.2553	0.2724
0.0025	1.4636	3350	0.0031	94803629.3273	128221.0494	11241.7382	679.7643	0.2651	0.2880
0.0025	1.4854	3400	0.0031	94396410.7443	127179.9998	11186.0052	677.2300	0.2513	0.2666
0.0025	1.5073	3450	0.0031	93961675.3576	124644.5782	11150.4429	677.7115	0.2542	0.2698
0.0024	1.5291	3500	0.0031	94294944.7468	127062.6771	11169.9988	678.8583	0.2607	0.2791
0.0026	1.5510	3550	0.0031	94680207.2586	131316.1834	11181.2686	678.1171	0.2591	0.2780
0.0026	1.5728	3600	0.0031	94565395.6252	130618.5136	11172.3922	679.2413	0.2583	0.2796
0.0027	1.5947	3650	0.0031	94494133.3128	130383.0332	11174.0985	676.0815	0.2586	0.2782
0.0024	1.6165	3700	0.0031	94479563.4830	130491.3072	11167.0927	676.8210	0.2573	0.2767
0.0029	1.6384	3750	0.0030	94238366.4133	130277.7676	11137.1134	676.0669	0.2563	0.2756
0.0026	1.6602	3800	0.0030	93864199.7122	127162.4395	11110.5482	677.2280	0.2612	0.2804
0.0026	1.6821	3850	0.0030	93756032.3397	126485.2058	11105.6547	675.8117	0.2605	0.2806
0.0028	1.7039	3900	0.0030	93963130.6575	126515.3114	11135.2790	676.9840	0.2598	0.2802
0.0026	1.7258	3950	0.0031	93961082.1699	127061.8487	11139.1269	675.3646	0.2545	0.2717
0.0028	1.7476	4000	0.0030	93998847.4117	127969.0715	11132.3530	675.5727	0.2569	0.2763
0.0029	1.7694	4050	0.0030	93947498.7480	127780.3334	11127.4591	675.4349	0.2559	0.2745
0.0027	1.7913	4100	0.0030	93894907.3807	126891.6910	11127.1123	675.1372	0.2569	0.2755
0.0025	1.8131	4150	0.0030	94174406.7842	127694.4294	11158.2497	676.1042	0.2582	0.2783
0.0024	1.8350	4200	0.0030	93932226.1485	127238.5370	11127.9816	675.1781	0.2573	0.2761
0.0029	1.8568	4250	0.0030	93792415.1564	126753.9871	11110.6139	675.0652	0.2580	0.2771
0.0025	1.8787	4300	0.0030	93934620.2837	128020.7799	11118.6802	674.6389	0.2566	0.2757
0.0026	1.9005	4350	0.0030	94005246.6032	128531.9098	11125.6335	674.2449	0.2557	0.2746
0.0025	1.9224	4400	0.0030	93962324.3167	128205.5150	11121.7361	674.5847	0.2568	0.2765
0.0025	1.9442	4450	0.0030	93967490.9505	128247.8035	11121.3914	674.7800	0.2575	0.2773
0.0026	1.9661	4500	0.0030	93944799.2628	128244.6731	11118.9372	674.5379	0.2570	0.2765
0.0027	1.9879	4550	0.0030	93936054.2243	128110.2817	11118.5170	674.6631	0.2572	0.2767

Framework versions

Transformers 5.0.0
Pytorch 2.10.0+cu128
Datasets 4.0.0
Tokenizers 0.22.2

Downloads last month: 21

Safetensors

Model size

4B params

Tensor type

BF16

Model tree for namezz/lvm-a-qwen2.5-vl-7b-instruct-b-qwen2.5-vl-3b-instruct

Base model

Qwen/Qwen2.5-VL-3B-Instruct

Finetuned

(732)

this model

Collection including namezz/lvm-a-qwen2.5-vl-7b-instruct-b-qwen2.5-vl-3b-instruct

Length Value Model

Collection

11 items • Updated about 3 hours ago