Interested in Jackrong versions
I have been following the project from Jeff Geerling's video and your models have inspired me to setup an ai cluster locally and I'm grateful for the work you guys put in.
However I've tried the Opus Distill versions of Qwen 3.5 models and I found them to be smarter than the base versions. Would there be a possibility of having those models be optimised for Pi5s too? That would make the stack quite amazing.
Also gemma 4 26ba4 seems like an interesting model that I hope your team can look into.
Thank you for your work and contribution once again.
Thanks a lot, really appreciate it. Glad our work helped inspire your local AI setup.
We have another Qwen 3.5 coming very soon. We definitely have those popular models in mind and are trying to release the most useful ones for local AI enthusiasts.
That’s great to hear. I’ve been testing the Opus distills on constrained setups and they’ve been surprisingly strong, especially for more agentic workloads.
I was able to get Qwen3 30B A3 coder (Q3_K_S) running via a TurboQuant fork of llama.cpp at 16k context. I’ll be running structured tests on it and sharing results in the model’s discussion section.
I’m also exploring how these models behave in agent frameworks like Hermes and OpenClaw, with planned integrations like SIP and RAG to evaluate real-time smoothness across the full pipeline.
Would be really interesting to see how your upcoming Qwen 3.5 variants, especially the Opus distills, perform in these scenarios.