Settings
#2
by jeffbenson1979 - opened
Hello, just want to ask u 1 qst ...
I have 8X96GB GPUs , and when i run heretic , i see 5-6 mins 1 trial going, is it ok for 27B dense model? Bcs 122B 1min , 9B -20 sec. i dono if its ok with heretic or i do something wrong
Hello, just want to ask u 1 qst ...
I have 8X96GB GPUs , and when i run heretic , i see 5-6 mins 1 trial going, is it ok for 27B dense model? Bcs 122B 1min , 9B -20 sec. i dono if its ok with heretic or i do something wrong
No it shouldn't take so long,
i fixed my problem. model config.json should have use_cache : true..i got false