When using ollama to run this model, the output is "onse", you can use the TEMPLATE below to correct it
#6
by hejiangda - opened
create a new modelfile using the content below:
FROM hf.co/tencent/HY-MT1.5-1.8B-GGUF:Q8_0
TEMPLATE """{{ if .System }}<|startoftext|>{{ .System }}<|extra_4|>{{ end }}{{ if .Prompt }}<|startoftext|>{{ .Prompt }}<|extra_0|>{{ end }}{{ .Response }}<|eos|>"""
Last weekend I tried Ollama with this model. It's too slow in my MacBook M4 Pro. Could be this reason. I may try this tomorrow.
I finally find a new inference tool paddlerto serve as api backend of my immersive translation extension. https://medium.com/@jason.ni.py/use-hunyuan-translation-model-v1-5-1-8b-for-immersivetranslate-extension-4ab1bcd10478