Gemma 4 26B (A4B) - Garbage output (korean/chinese) on MLX 0.31.2 while 31B works fine
#2
by Z3NN001 - opened
My setup is a Mac Mini M4 (32GB RAM) running MLX 0.31.2. Here is the weird part: I can run the Gemma-2-31B CRACK versions perfectly, but this 26B version is just not working.
The issues:
When using vmlx, I get: ModuleNotFoundError: No module named 'jang_tools'. I've searched everywhere but can't find this package/plugin.
When trying mlx_vlm, I get: ValueError: Received 1695 parameters not in model (referencing layer_scalar, biases, and scales).
If I manage to force a "simple" load, the model speaks "martian" (garbage tokens, Korean/Chinese characters, and random symbols).
probably im doing something wrong, can someone help ?
I encountered the same problem, unable to solve it, and don't know why.