What is the purpose of the model?
#1
by NeObr - opened
This model was created for use on cell phones, which explains its size. Could you explain the usefulness of such a small model? Thank you for your work.
drafting
zhijianliu changed discussion status to closed
This model was created for use on cell phones, which explains its size. Could you explain the usefulness of such a small model? Thank you for your work.
It's a (mask diffusion based) drafting head attached to the main model for speculative decoding usage, please read related paper: https://arxiv.org/abs/2602.06036
And it's DEFINITELY NOT FOR CELL PHONES, unless your cell phone do have ~80GiB VMEM...