Is the config,json correct?
#2
by HankFromResearch - opened
is "max_position_embeddings": 2048 correct?
I only ask this because I have noticed that some updates in other GPTQ repos have changed max_position_embeddings to 4096.
I just set my position embedding compression factor to 2 (max length is 4096), since I just assumed it was a particular nuance of this quant.
Yeah you're right it should be 4096. The original Llama 2 models set 2048 so the fine tunes that came out early also had 2048. But it should be 4096 for all Llama 2 models and Llama 2 fine tunes
I'll fix that now
Awesome, love your work, cheers for that.
HankFromResearch changed discussion status to closed