original_max_position_embeddings for YaRN

#44
by XirenZhou - opened

In the model card/README, why is it
--rope-scaling '{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":32768}' --max-model-len 131072
instead of
--rope-scaling '{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":40960}' --max-model-len 163840 ?

Sign up or log in to comment