mlx-community

/

Llama-3_3-Nemotron-Super-49B-v1-mlx-4bit

like 2

Follow

MLX Community 11.6k

Same situation here (LM Studio Version: 0.3.16 / MLX engine 0.17.0): Error in iterating prediction stream: AttributeError: 'NoneType' object has no attribute 'shape'

The bug arises because in Jinja the filter operator (|) has higher precedence than the string‐concatenation operator (+). In the faulty line:

'…' + content | trim + '…'

only content is trimmed—Jinja interprets it as:

'…' + (content | trim) + '…'

Consequently, the surrounding tokens aren’t included in the trim, and the final '<|eot_id|>' can end up misplaced.

Fix: Wrap the entire concatenated string in parentheses and then apply | trim. For example:

( '<|start_header_id|>' + message.role + '<|end_header_id|>\n\n' + content + '<|eot_id|>' ) | trim

This ensures Jinja first builds the full string (header, role, content, footer) and only then removes leading/trailing whitespace.

I'm also facing the same error, but wasn't able to fix it following your explanation above. Could you please share the complete (correct) ninja template?

Hi!
We’ve struggled a lot with Nemotron Ultra conversion and inference settings adjustments but we succeed.
Try to modify the modified jinja template and a tokenizer config files adjustment we did in ours https://huggingface.co/LibraxisAI/Llama-3_1-Nemotron-Ultra-253B-v1-MLX-Q5.
If you have any questions I am open to help.
Maciej

I set up the code in the linked chat_template.jinja in lm studio, but I get a parse error!

Can someone please share the complete jinja code available in lm studio?