Trying to run in a google notebook bartowski/Qwen2.5-32B-Instruct-GGUF
#3
by Javedalam - opened
It kind of run but I have not been able to get the output right. I get a truncated output
The Colab notebook is here
https://colab.research.google.com/drive/1aBA3KarQLpqRj8NU6Xhliur30tK887Mn?usp=sharing
REPO = "bartowski/DeepSeek-R1-Distill-Qwen-32B-GGUF" # more explicit reasoning
FILENAME = "DeepSeek-R1-Distill-Qwen-32B-Q4_K_M.gguf"