gsaltintas commited on
Commit
cd7535d
·
verified ·
1 Parent(s): 2e3791c

Upload tokenizer file flexitok--mod-tokenizers-individual_overlap.json - Upload model files

Browse files
flexitok--mod-tokenizers-individual_overlap.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"9": {"ratio_to_total_tokens": 1.0, "expected_training_ratio_in_superset": 1.0, "num_tokens": 17}, "total_training_compared_to_full_model": 1.0}