boffire
/

Kabyle-BPE-Tokenizer-v2

Text Generation

Model card Files Files and versions

Kabyle-BPE-Tokenizer-v2 / special_tokens_map.json

boffire's picture

Upload 3 files

4941498 verified about 8 hours ago

history blame contribute delete

121 Bytes

	{
	"bos_token": "<s>",
	"eos_token": "</s>",
	"unk_token": "<unk>",
	"pad_token": "<pad>",
	"mask_token": "<mask>"
	}