Xenova

/

esm2_t6_8M_UR50D

like 0

Hey @Xenova 👋,

Thanks for creating this model. I'm trying to run it in a browser but unfortunately I'm getting the following error Error: Mask token (null) not found in text.

Here's some sample code to reproduce the error:

import { pipeline, env } from 'https://cdn.jsdelivr.net/npm/@xenova/transformers@2.17.2';
env.allowLocalModels = false;
let pipe = await pipeline("fill-mask", "Xenova/esm2_t6_8M_UR50D");
let out = await pipe(`${pipe.tokenizer.mask_token}YIQNCPLG`);

Printing pipe.tokenizer.mask_token reveals that it is null but it should be <mask>. Perhaps there's an issue with tokenizer.json or tokenizer_config.json?