Update processor_config.json
#81
by Akshay47 - opened
Add unk_token to processor_config.json
This update adds an "unk_token" field to improve handling of unknown tokens during processing, enhancing robustness and compatibility with tokenizer settings.