RuntimeError: Expected all tensors to be on the same device
#3
by SekiroRong - opened
Bravo for your great work! However, do you have any idea about the RuntimeError when infer after apply your Qint4 model (use load_quantized_hi3_m2)?
Please try select a image size, for example 1:1, don't use "Auto"; and System Prompt "None", Bot Task "Image", Context Mode "Single Round";
the List of Model input info: like below should ok:
This is probably related to the official inferencing code, should be not related to the model itself.
It works, thank you!
wikeeyang changed discussion status to closed