Tokenizer Study (LLaMA 350M) Collection Correlating tokenizer properties of pre-trained LLMs with their downstream performance. • 33 items • Updated Mar 17
shikhar-srivastava/mono_350m_pre_lr1e-3_fineweb2_tha_unigram_unscaled_65536 0.4B • Updated Mar 16 • 8
shikhar-srivastava/mono_350m_pre_lr1e-3_fineweb2_tha_unigram_unscaled_65536 0.4B • Updated Mar 16 • 8
shikhar-srivastava/mono_350m_pre_lr1e-3_fineweb2_vie_unigram_unscaled_65536 0.4B • Updated Mar 16 • 6