shikhar-srivastava/llama-130m-prenorm-train_c4-2B-tok_t5base
Updated • 1
Tokenizer Study (LLaMA 130M)
Correlating tokenizer properties on pre-trained LLMs with their downstream performance.
-
-
shikhar-srivastava/llama-130m-prenorm-train_c4_2B-tok_llama2
0.1B • Updated • 5 -
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_eng_latn_bpe_unscaled_8192
97.5M • Updated • 857 -
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_eng_latn_unigram_unscaled_8192
97.5M • Updated • 251 -
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_tha_thai_bpe_unscaled_8192
97.5M • Updated • 2 -
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_tha_thai_unigram_unscaled_8192
97.5M • Updated • 1 -
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_urd_arab_bpe_unscaled_8192
97.5M • Updated • 1 -
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_vie_latn_bpe_unscaled_8192
97.5M • Updated • 1 -
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_amh_ethi_bpe_unscaled_8192
97.5M • Updated • 1
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_tha_thai_bpe_unscaled_16384
0.1B • Updated • 1Note Model with unscaled script, bpe tokenizer, vocab size 16384
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_tha_thai_bpe_unscaled_32768
0.1B • Updated • 130Note Model with unscaled script, bpe tokenizer, vocab size 32768
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_tha_thai_bpe_unscaled_49152
0.2B • Updated • 1Note Model with unscaled script, bpe tokenizer, vocab size 49152
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_tha_thai_bpe_unscaled_65536
0.2B • Updated • 3Note Model with unscaled script, bpe tokenizer, vocab size 65536
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_tha_thai_bpe_unscaled_81920
0.2B • Updated • 3Note Model with unscaled script, bpe tokenizer, vocab size 81920
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_tha_thai_bpe_unscaled_98304
0.2B • Updated • 1Note Model with unscaled script, bpe tokenizer, vocab size 98304
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_urd_arab_bpe_unscaled_16384
0.1B • Updated • 1Note Model with unscaled script, bpe tokenizer, vocab size 16384
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_urd_arab_bpe_unscaled_32768
0.1B • Updated • 1Note Model with unscaled script, bpe tokenizer, vocab size 32768
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_urd_arab_bpe_unscaled_49152
0.2B • Updated • 1Note Model with unscaled script, bpe tokenizer, vocab size 49152
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_urd_arab_bpe_unscaled_65536
0.2B • Updated • 1Note Model with unscaled script, bpe tokenizer, vocab size 65536
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_urd_arab_bpe_unscaled_81920
0.2B • Updated • 1Note Model with unscaled script, bpe tokenizer, vocab size 81920
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_urd_arab_bpe_unscaled_98304
0.2B • Updated • 1Note Model with unscaled script, bpe tokenizer, vocab size 98304
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_amh_ethi_bpe_unscaled_16384
0.1B • Updated • 1Note Model with unscaled script, bpe tokenizer, vocab size 16384
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_amh_ethi_bpe_unscaled_32768
0.1B • Updated • 1Note Model with unscaled script, bpe tokenizer, vocab size 32768
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_amh_ethi_bpe_unscaled_49152
0.2B • Updated • 1Note Model with unscaled script, bpe tokenizer, vocab size 49152
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_amh_ethi_bpe_unscaled_65536
0.2B • Updated • 1Note Model with unscaled script, bpe tokenizer, vocab size 65536
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_amh_ethi_bpe_unscaled_81920
0.2B • Updated • 2Note Model with unscaled script, bpe tokenizer, vocab size 81920
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_amh_ethi_bpe_unscaled_98304
0.2B • Updated • 1Note Model with unscaled script, bpe tokenizer, vocab size 98304
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_vie_latn_bpe_unscaled_16384
0.1B • Updated • 1Note Model with unscaled script, bpe tokenizer, vocab size 16384
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_vie_latn_bpe_unscaled_32768
0.1B • Updated • 1Note Model with unscaled script, bpe tokenizer, vocab size 32768
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_vie_latn_bpe_unscaled_49152
0.2B • Updated • 1Note Model with unscaled script, bpe tokenizer, vocab size 49152
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_vie_latn_bpe_unscaled_65536
0.2B • Updated • 1Note Model with unscaled script, bpe tokenizer, vocab size 65536
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_vie_latn_bpe_unscaled_81920
0.2B • Updated • 1Note Model with unscaled script, bpe tokenizer, vocab size 81920
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_eng_latn_bpe_unscaled_16384
0.1B • Updated • 529Note Model with unscaled script, bpe tokenizer, vocab size 16384
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_eng_latn_bpe_unscaled_32768
0.1B • Updated • 647Note Model with unscaled script, bpe tokenizer, vocab size 32768
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_eng_latn_bpe_unscaled_49152
0.2B • Updated • 398Note Model with unscaled script, bpe tokenizer, vocab size 49152
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_eng_latn_bpe_unscaled_65536
0.2B • Updated • 377Note Model with unscaled script, bpe tokenizer, vocab size 65536
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_eng_latn_bpe_unscaled_81920
0.2B • Updated • 378Note Model with unscaled script, bpe tokenizer, vocab size 81920
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_eng_latn_bpe_unscaled_98304
0.2B • Updated • 380Note Model with unscaled script, bpe tokenizer, vocab size 98304
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_eng_latn_bpe_unscaled_114688
0.3B • Updated • 351Note Model with unscaled script, bpe tokenizer, vocab size 114688
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_eng_latn_bpe_unscaled_262144
Updated • 80Note Model with unscaled script, bpe tokenizer, vocab size 262144
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_tha_thai_bpe_unscaled_114688
0.3B • Updated • 1Note Model with unscaled script, bpe tokenizer, vocab size 114688
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_urd_arab_bpe_unscaled_114688
0.3B • Updated • 1Note Model with unscaled script, bpe tokenizer, vocab size 114688
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_urd_arab_bpe_unscaled_262144
0.5B • Updated • 1Note Model with unscaled script, bpe tokenizer, vocab size 262144
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_amh_ethi_bpe_unscaled_114688
0.3B • Updated • 1Note Model with unscaled script, bpe tokenizer, vocab size 114688
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_tha_thai_unigram_unscaled_16384
0.1B • Updated • 4Note Model with unscaled script, unigram tokenizer, vocab size 16384
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_tha_thai_unigram_unscaled_32768
0.1B • Updated • 6Note Model with unscaled script, unigram tokenizer, vocab size 32768
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_tha_thai_unigram_unscaled_49152
0.2B • Updated • 1Note Model with unscaled script, unigram tokenizer, vocab size 49152
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_tha_thai_unigram_unscaled_65536
0.2B • Updated • 1Note Model with unscaled script, unigram tokenizer, vocab size 65536
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_tha_thai_unigram_unscaled_81920
0.2B • Updated • 1Note Model with unscaled script, unigram tokenizer, vocab size 81920
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_tha_thai_unigram_unscaled_98304
0.2B • Updated • 1Note Model with unscaled script, unigram tokenizer, vocab size 98304
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_tha_thai_unigram_unscaled_114688
0.3B • Updated • 1Note Model with unscaled script, unigram tokenizer, vocab size 114688
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_eng_latn_unigram_unscaled_16384
0.1B • Updated • 253Note Model with unscaled script, unigram tokenizer, vocab size 16384
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_eng_latn_unigram_unscaled_32768
0.1B • Updated • 235Note Model with unscaled script, unigram tokenizer, vocab size 32768
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_eng_latn_unigram_unscaled_49152
0.2B • Updated • 132Note Model with unscaled script, unigram tokenizer, vocab size 49152
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_eng_latn_unigram_unscaled_65536
0.2B • Updated • 130Note Model with unscaled script, unigram tokenizer, vocab size 65536
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_eng_latn_unigram_unscaled_81920
0.2B • Updated • 78Note Model with unscaled script, unigram tokenizer, vocab size 81920
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_eng_latn_unigram_unscaled_98304
0.2B • Updated • 2Note Model with unscaled script, unigram tokenizer, vocab size 98304
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_eng_latn_unigram_unscaled_114688
0.3B • Updated • 2Note Model with unscaled script, unigram tokenizer, vocab size 114688
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_tha_thai_bpe_unscaled_262144
0.5B • Updated • 6Note Model with unscaled script, bpe tokenizer, vocab size 262144
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_urd_arab_unigram_unscaled_8192
97.5M • UpdatedNote Model with unscaled script, unigram tokenizer, vocab size 8192
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_urd_arab_unigram_unscaled_16384
0.1B • Updated • 1Note Model with unscaled script, unigram tokenizer, vocab size 16384
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_urd_arab_unigram_unscaled_32768
0.1B • Updated • 1Note Model with unscaled script, unigram tokenizer, vocab size 32768
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_urd_arab_unigram_unscaled_49152
0.2B • Updated • 1Note Model with unscaled script, unigram tokenizer, vocab size 49152
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_urd_arab_unigram_unscaled_65536
0.2B • Updated • 1Note Model with unscaled script, unigram tokenizer, vocab size 65536
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_amh_ethi_unigram_unscaled_8192
97.5M • UpdatedNote Model with unscaled script, unigram tokenizer, vocab size 8192
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_amh_ethi_unigram_unscaled_16384
0.1B • Updated • 1Note Model with unscaled script, unigram tokenizer, vocab size 16384
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_amh_ethi_unigram_unscaled_32768
0.1B • Updated • 3Note Model with unscaled script, unigram tokenizer, vocab size 32768
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_amh_ethi_unigram_unscaled_49152
0.2B • Updated • 1Note Model with unscaled script, unigram tokenizer, vocab size 49152
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_amh_ethi_unigram_unscaled_65536
0.2B • Updated • 1Note Model with unscaled script, unigram tokenizer, vocab size 65536
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_amh_ethi_unigram_unscaled_81920
0.2B • Updated • 1Note Model with unscaled script, unigram tokenizer, vocab size 81920
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_amh_ethi_bpe_unscaled_262144
0.5B • Updated • 1Note Model with unscaled script, bpe tokenizer, vocab size 262144
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_vie_latn_unigram_unscaled_8192
97.5M • UpdatedNote Model with unscaled script, unigram tokenizer, vocab size 8192
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_vie_latn_unigram_unscaled_16384
0.1B • Updated • 1Note Model with unscaled script, unigram tokenizer, vocab size 16384
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_vie_latn_unigram_unscaled_32768
0.1B • Updated • 1Note Model with unscaled script, unigram tokenizer, vocab size 32768
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_vie_latn_unigram_unscaled_49152
0.2B • Updated • 1Note Model with unscaled script, unigram tokenizer, vocab size 49152
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_vie_latn_unigram_unscaled_65536
0.2B • Updated • 1Note Model with unscaled script, unigram tokenizer, vocab size 65536
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_vie_latn_unigram_unscaled_81920
0.2B • Updated • 1Note Model with unscaled script, unigram tokenizer, vocab size 81920
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_vie_latn_bpe_unscaled_98304
0.2B • Updated • 3Note Model with unscaled script, bpe tokenizer, vocab size 98304
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_vie_latn_unigram_unscaled_98304
0.2B • Updated • 1Note Model with unscaled script, unigram tokenizer, vocab size 98304
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_vie_latn_bpe_unscaled_114688
0.3B • Updated • 1Note Model with unscaled script, bpe tokenizer, vocab size 114688
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_vie_latn_unigram_unscaled_114688
0.3B • Updated • 1Note Model with unscaled script, unigram tokenizer, vocab size 114688
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_vie_latn_bpe_unscaled_262144
0.5B • Updated • 1Note Model with unscaled script, bpe tokenizer, vocab size 262144
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_urd_arab_unigram_unscaled_81920
0.2B • Updated • 6Note Model with unscaled script, unigram tokenizer, vocab size 81920
shikhar-srivastava/mono_gold_130m_pre_lr1e-4_amh_ethi_unigram_unscaled_98304
0.2B • Updated • 1Note Model with unscaled script, unigram tokenizer, vocab size 98304