Infinity conversation, sould eos be <|eot_id|> ?
#1
by Komposter43 - opened
Model doesn't stop in vllm chat conversation, may be eos should be '<|eot_id|>' ?
put both eos tokens as stopping criteria i`d suggest.
See related merged PR on the meta-llama repo https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct/discussions/4. That change to the tokenizer basically just needs to be done here as well
Fixed!
casperhansen changed discussion status to closed