drama-base / tokenizer.json
ccsasuke's picture
Integrate Sentence Transformers, prevent manual tokenizer EOS (#1)
1d0338c verified
This file is stored with Xet . It is too big to display, but you can still download it.

Xet Pointer Details

( Raw pointer file )
Xet hash:
0768dde05472bdbb3aa9df441a9bb77b31da272aca763461edfb98146b9432f5
Size of remote file:
17.2 MB
·
SHA256:
6c18e1797510535655f962df0669fcb7d10b325b5d0eb4b51be36789dcf5fcaf

Xet efficiently stores Large Files inside Git, intelligently splitting files into unique chunks and accelerating uploads and downloads. More info.