indus-script-models / data /indus_tokenizer /tokenizer_info.json
hellosindh's picture
Upload 4 files
106df91 verified
{
"vocab_size": 646,
"indus_token_count": 641,
"pad_token_id": 0,
"cls_token_id": 2,
"sep_token_id": 3,
"mask_token_id": 4,
"unk_token_id": 1
}