questions about model architecture
#1
by Advik-7 - opened
hey , which model architecture does this nano version follow !?! not much information is available of this
No idea, this was accidentally released a while back without any info. I just wanted to see if I could do GGUFs for it and it worked for some reason but couldn't actually run it. So this should probably be removed.
lol alright!! makes sense , i explored a bit and it's made using Custom Llama class , they've reduced the token space of the model so it's not exactly Orpheus but a different model architecture distill attempt which probably failed