Create tokenizer.json
#1
by Jilt - opened
tokenizer.json will be required for conversions (for example to CT2 format). It also makes it a consistent file type across different whisper models.
Thanks for this
ylacombe changed pull request status to merged