Hugging Face's logo

IDMedicine
/
code-graph-trajeval-v1

Model card Files
xet
 Community
Bremin commited on
Commit
a38a118
·
verified ·
1 Parent(s): b10ebff

Upload harvest.log with huggingface_hub

Browse files
Files changed (1) hide show
  1. harvest.log +72 -0
harvest.log ADDED
@@ -0,0 +1,72 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [harvest] split=verified loading task metadata ...
2
+ [harvest] 500 task entries in 2.1s
3
+ [harvest] harvesting 5 submissions: ['20240620_sweagent_claude3.5sonnet', '20241022_tools_claude-3-5-sonnet-updated', '20241028_agentless-1.5_gpt4o', '20241029_OpenHands-CodeAct-2.1-sonnet-20241022', '20250405_amazon-q-developer-agent-20250405-dev']
4
+ [harvest-s3] listing verified/20240620_sweagent_claude3.5sonnet/logs/ ...
5
+ [harvest-s3] 489 instance dirs
6
+ [harvest-s3] 50/489 (16.5/s, kept=50)
7
+ [harvest-s3] 100/489 (18.7/s, kept=100)
8
+ [harvest-s3] 150/489 (19.6/s, kept=149)
9
+ [harvest-s3] 200/489 (18.4/s, kept=198)
10
+ [harvest-s3] 250/489 (23.5/s, kept=248)
11
+ [harvest-s3] 300/489 (22.6/s, kept=297)
12
+ [harvest-s3] 350/489 (21.7/s, kept=347)
13
+ [harvest-s3] 400/489 (22.6/s, kept=396)
14
+ [harvest-s3] 450/489 (22.4/s, kept=446)
15
+ [harvest-s3] 20240620_sweagent_claude3.5sonnet: kept 485 (resolved=168, rate=34.6%)
16
+ [harvest-s3] listing verified/20241022_tools_claude-3-5-sonnet-updated/logs/ ...
17
+ [harvest-s3] 486 instance dirs
18
+ [harvest-s3] 50/486 (15.7/s, kept=50)
19
+ [harvest-s3] 100/486 (17.6/s, kept=100)
20
+ [harvest-s3] 150/486 (18.0/s, kept=150)
21
+ [harvest-s3] 200/486 (17.7/s, kept=200)
22
+ [harvest-s3] 250/486 (17.9/s, kept=250)
23
+ [harvest-s3] 300/486 (20.8/s, kept=299)
24
+ [harvest-s3] 350/486 (20.8/s, kept=349)
25
+ [harvest-s3] 400/486 (19.2/s, kept=397)
26
+ [harvest-s3] 450/486 (20.1/s, kept=447)
27
+ [harvest-s3] 20241022_tools_claude-3-5-sonnet-updated: kept 483 (resolved=245, rate=50.7%)
28
+ [harvest-s3] listing verified/20241028_agentless-1.5_gpt4o/logs/ ...
29
+ [harvest-s3] 496 instance dirs
30
+ [harvest-s3] 50/496 (16.2/s, kept=50)
31
+ [harvest-s3] 100/496 (17.1/s, kept=100)
32
+ [harvest-s3] 150/496 (16.7/s, kept=150)
33
+ [harvest-s3] 200/496 (18.6/s, kept=200)
34
+ [harvest-s3] 250/496 (18.9/s, kept=250)
35
+ [harvest-s3] 300/496 (18.5/s, kept=300)
36
+ [harvest-s3] 350/496 (18.1/s, kept=350)
37
+ [harvest-s3] 400/496 (21.2/s, kept=400)
38
+ [harvest-s3] 450/496 (20.9/s, kept=449)
39
+ [harvest-s3] 20241028_agentless-1.5_gpt4o: kept 495 (resolved=194, rate=39.2%)
40
+ [harvest-s3] listing verified/20241029_OpenHands-CodeAct-2.1-sonnet-20241022/logs/ ...
41
+ [harvest-s3] 494 instance dirs
42
+ [harvest-s3] 50/494 (15.0/s, kept=50)
43
+ [harvest-s3] 100/494 (15.3/s, kept=100)
44
+ [harvest-s3] 150/494 (18.0/s, kept=149)
45
+ [harvest-s3] 200/494 (18.6/s, kept=199)
46
+ [harvest-s3] 250/494 (18.5/s, kept=249)
47
+ [harvest-s3] 300/494 (18.0/s, kept=299)
48
+ [harvest-s3] 350/494 (17.6/s, kept=349)
49
+ [harvest-s3] 400/494 (19.4/s, kept=399)
50
+ [harvest-s3] 450/494 (19.8/s, kept=449)
51
+ [harvest-s3] 20241029_OpenHands-CodeAct-2.1-sonnet-20241022: kept 493 (resolved=265, rate=53.8%)
52
+ [harvest-s3] listing verified/20250405_amazon-q-developer-agent-20250405-dev/logs/ ...
53
+ [harvest-s3] 500 instance dirs
54
+ [harvest-s3] 50/500 (11.8/s, kept=50)
55
+ [harvest-s3] 100/500 (17.8/s, kept=100)
56
+ [harvest-s3] 150/500 (18.4/s, kept=150)
57
+ [harvest-s3] 200/500 (18.8/s, kept=200)
58
+ [harvest-s3] 250/500 (19.2/s, kept=250)
59
+ [harvest-s3] 300/500 (18.7/s, kept=300)
60
+ [harvest-s3] 350/500 (18.4/s, kept=350)
61
+ [harvest-s3] 400/500 (18.8/s, kept=400)
62
+ [harvest-s3] 450/500 (19.0/s, kept=450)
63
+ [harvest-s3] 500/500 (20.7/s, kept=500)
64
+ [harvest-s3] 20250405_amazon-q-developer-agent-20250405-dev: kept 500 (resolved=330, rate=66.0%)
65
+
66
+ [harvest] wrote 2456 trajs → outputs/traj_real/specs.jsonl
67
+ [harvest] resolved overall: 1202/2456 (48.9%)
68
+ 20240620_sweagent_claude3.5sonnet n= 485 resolved= 168 rate=34.6%
69
+ 20241022_tools_claude-3-5-sonnet-updated n= 483 resolved= 245 rate=50.7%
70
+ 20241028_agentless-1.5_gpt4o n= 495 resolved= 194 rate=39.2%
71
+ 20241029_OpenHands-CodeAct-2.1-sonnet-20241022 n= 493 resolved= 265 rate=53.8%
72
+ 20250405_amazon-q-developer-agent-20250405-dev n= 500 resolved= 330 rate=66.0%