ARFBench

Running

sxie78-dd commited on 16 days ago

Commit

94c4135

unverified ·

1 Parent(s): 50e3fda

update naming

Files changed (3) hide show

results/ARFBench_leaderboard.csv CHANGED Viewed

@@ -17,6 +17,6 @@ GPT-5.4,VLM,61.3,81.1,54.2,61.3,51.4,62.6,50.4,48.4
 GPT-5,VLM,62.7,82.0,55.9,62.5,51.9,66.9,51.2,47.5
 OpenTSLM 1B (TS-LLM),Post-trained TSFM,0.8,0.0,2.0,0.0,1.2,0.0,3.0,0.0
 ChatTS 8B (TS-LLM),Post-trained TSFM,31.1,60.4,26.5,25.5,22.1,48.1,20.0,15.4
-Toto-Qwen3 32B (TSFM-LLM),Post-trained TSFM,48.8,82.9,47.4,38.7,33.9,60.0,43.6,16.4
 Qwen3-VL 32B (post-trained),Post-trained TSFM,56.9,84.7,50.3,53.8,46.6,69.8,44.9,40.5
-Toto-VLM 32B (TSFM-VLM),Post-trained TSFM,63.9,84.7,55.6,64.6,48.9,66.3,48.4,43.5

 GPT-5,VLM,62.7,82.0,55.9,62.5,51.9,66.9,51.2,47.5
 OpenTSLM 1B (TS-LLM),Post-trained TSFM,0.8,0.0,2.0,0.0,1.2,0.0,3.0,0.0
 ChatTS 8B (TS-LLM),Post-trained TSFM,31.1,60.4,26.5,25.5,22.1,48.1,20.0,15.4
+Toto-1.0-Qwen3 32B (TSFM-LLM),Post-trained TSFM,48.8,82.9,47.4,38.7,33.9,60.0,43.6,16.4
 Qwen3-VL 32B (post-trained),Post-trained TSFM,56.9,84.7,50.3,53.8,46.6,69.8,44.9,40.5
+Toto-1.0-QA-Experimental 32B (TSFM-VLM),Post-trained TSFM,63.9,84.7,55.6,64.6,48.9,66.3,48.4,43.5

results/ARFBench_leaderboard_category_accuracy.csv CHANGED Viewed

@@ -17,6 +17,6 @@ GPT-5.4,VLM,61.3,81.1,31.6,63.6,65.6,57.9,56.7,61.8,60.7
 GPT-5,VLM,62.7,82.0,31.6,44.6,68.8,65.8,59.6,63.5,61.3
 OpenTSLM (TS-LLM),Post-trained TSFM,0.8,0.0,0.0,3.6,0.0,5.3,0.0,0.0,0.0
 ChatTS (TS-LLM),Post-trained TSFM,31.1,59.5,15.8,16.1,15.6,28.9,20.2,40.0,14.7
-Toto-Qwen3 (TSFM-LLM),Post-trained TSFM,48.8,82.9,10.5,35.7,34.4,47.4,71.2,41.8,35.6
 Qwen3-VL 32B (post-trained),Post-trained TSFM,56.9,84.7,36.8,41.1,43.8,63.2,52.9,67.6,39.3
-Toto-VLM (TSFM-VLM),Post-trained TSFM,63.9,84.7,47.4,26.8,59.4,64.5,66.3,68.8,60.1

 GPT-5,VLM,62.7,82.0,31.6,44.6,68.8,65.8,59.6,63.5,61.3
 OpenTSLM (TS-LLM),Post-trained TSFM,0.8,0.0,0.0,3.6,0.0,5.3,0.0,0.0,0.0
 ChatTS (TS-LLM),Post-trained TSFM,31.1,59.5,15.8,16.1,15.6,28.9,20.2,40.0,14.7
+Toto-1.0-Qwen3 (TSFM-LLM),Post-trained TSFM,48.8,82.9,10.5,35.7,34.4,47.4,71.2,41.8,35.6
 Qwen3-VL 32B (post-trained),Post-trained TSFM,56.9,84.7,36.8,41.1,43.8,63.2,52.9,67.6,39.3
+Toto-1.0-QA-Experimental (TSFM-VLM),Post-trained TSFM,63.9,84.7,47.4,26.8,59.4,64.5,66.3,68.8,60.1

results/ARFBench_leaderboard_category_f1.csv CHANGED Viewed

@@ -17,6 +17,6 @@ GPT-5.4,VLM,51.4,62.6,29.6,53.3,55.1,51.7,54.1,47.7,49.1
 GPT-5,VLM,51.9,66.8,32.8,44.2,47.8,59.1,57.0,49.0,45.9
 OpenTSLM 1B (TS-LLM),Post-trained TSFM,1.2,0.0,8.2,2.7,0.0,6.0,0.0,0.0,0.0
 ChatTS 8B (TS-LLM),Post-trained TSFM,22.1,48.1,22.2,15.0,14.4,27.9,17.9,21.4,9.2
-Toto-Qwen3 32B (TSFM-LLM),Post-trained TSFM,33.9,59.9,17.5,41.3,23.0,35.9,66.2,18.6,14.1
 Qwen3-VL 32B (post-trained),Post-trained TSFM,46.6,69.7,40.5,37.2,36.7,48.9,50.3,46.8,33.9
-Toto-VLM 32B (TSFM-VLM),Post-trained TSFM,48.9,66.3,46.9,23.0,48.8,54.1,58.4,44.2,42.7

 GPT-5,VLM,51.9,66.8,32.8,44.2,47.8,59.1,57.0,49.0,45.9
 OpenTSLM 1B (TS-LLM),Post-trained TSFM,1.2,0.0,8.2,2.7,0.0,6.0,0.0,0.0,0.0
 ChatTS 8B (TS-LLM),Post-trained TSFM,22.1,48.1,22.2,15.0,14.4,27.9,17.9,21.4,9.2
+Toto-1.0-Qwen3 32B (TSFM-LLM),Post-trained TSFM,33.9,59.9,17.5,41.3,23.0,35.9,66.2,18.6,14.1
 Qwen3-VL 32B (post-trained),Post-trained TSFM,46.6,69.7,40.5,37.2,36.7,48.9,50.3,46.8,33.9
+Toto-1.0-QA-Experimental 32B (TSFM-VLM),Post-trained TSFM,48.9,66.3,46.9,23.0,48.8,54.1,58.4,44.2,42.7