Thank you bartowski: q2,q3 can run on ryzen 3500u laptop.
#1
by BingoBird - opened
Very much a frontier experience on a small laptop.
I'm turning --reasoning-buget to 0 and running on a 16GB ryzen 3500u laptop with integrated GPU accessinng zram-compressed lpddr4.
It's an extreme setup and can only run q2-q3 quants at 1.7-4.5 t/s genenration. Looks like more work exploring inference-time params will be needed, as it's glitching a bit writing bash shellscript.
You can really see the craftsmanship that goes into the reasoning patterns nowaways!
Try use my last models, i use too a Ryzen whitout GPU Nvidia, Ryzen5 16gb only... use with full ctx kkkkkk