Will pretraining help?
#1
by rumourscape - opened
Thanks for sharing your work. I am also basically using the same approach like you but trying to pretrain first. Continued Pretraining on small data isn't working for me. Have you tried that or just directly finetuning is a better approach?
Thanks for your interest! Sorry but I don't speak English well :( the description is with automatic translation ... under the model