You should try training a model with 2B parameters and context length 32000.

by win10 - opened Dec 31, 2023

You should try training a model with 2B parameters and context length 32000.
Wishing you a happy new year and a successful new year.

I am more interested on why @PY007 settled on 1.1B? How did he/she come out with this number?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment