colab / kaggle
#5
by moon005 - opened
hi,
i am trying to run this in colab using stable diffusion cpp method and in the inference part i am unable to run it with gpu. when i remove —offload-to-cpu parameter, it still throws error although i have set already cuda.
can anyone share thier notebook or code sample on how to run this model ?
moon005 changed discussion status to closed
moon005 changed discussion status to open
figured it out. i just used --clip-on-cpu parameter to run Qwen2.5-VL-7B-Instruct-UD-Q4_K_XL.gguf on cpu and the rest can run on gpu.
moon005 changed discussion status to closed