If you want toggle for thinking for gguf's at LmStudio
For nohurry gguf's:
Go to your folder -> C:\Users\YOURNAME.lmstudio\hub\models\google\gemma-4-26B-A4B-it-heretic-GUFF (if you dont have last folders make them). Make file model.yaml
Open notepad and copy:
model.yaml is an open standard for defining cross-platform, composable AI models
Learn more at https://modelyaml.org
model: google/gemma-4-26B-A4B-it-heretic-GUFF
base:
- key: nohurry/gemma-4-26B-A4B-it-heretic-GUFF
sources:- type: huggingface
user: nohurry
repo: gemma-4-26B-A4B-it-heretic-GUFF
- type: huggingface
config:
operation:
fields:
- key: llm.prediction.temperature
value: 1.0
- key: llm.prediction.topPSampling
value:
checked: true
value: 0.95
- key: llm.prediction.topKSampling
value: 64
- key: llm.prediction.reasoning.parsing
value:
enabled: true
startString: "<|channel>thought"
endString: "<channel|>"
customFields:
- key: enableThinking
displayName: Enable Thinking
description: Controls whether the model will think before replying
type: boolean
defaultValue: true
effects:- type: setJinjaVariable
variable: enable_thinking
- type: setJinjaVariable
metadataOverrides:
domain: llm
architectures:
- gemma4
compatibilityTypes:
- gguf
paramsStrings:
- 26B
minMemoryUsageBytes: 17000000000
contextLengths:
- 262144
vision: true
reasoning: true
trainedForToolUse: true
I unfortunately couldn't get this to work, though it is probably a formatting issue. The formatting changes some of the symbols and indentations in your post, but using 'Quote' from the triple dots in the top right of your post can give your original text in a reply, and then removing the '> ' quotes at the start of each line can give the clean text.
Also added the slash after C:\Users\YOURNAME\ which might have gotten stripped.
Google suggested putting the yaml file in the gguf file location, though that caused LMStudio to not list the model at all.
edit: It might be because I didn't have the mmproj weights for vision. Grabbing those now and maybe it will prevent a silent failure which was happening when trying to add the model to the list.
edit 2: I had to rename the mmproj file to mmproj-gemma-4-26B-A4B-it-heretic.bf16.gguf to get it be detected by LM Studio and auto associated with gemma-4-26b-a4b-it-heretic.q4_k_m.gguf, and now Vision is automatically enabled.
Someone else make step by step tutorial at Reddit - https://www.reddit.com/r/LocalLLaMA/comments/1sc9s1x/tutorial_how_to_toggle_onoff_the_thinking_mode/
(but you need only 1 file - model.yaml)
Thanks! Once I added the mmproj vision weights with the correct name, and put the yaml file in the original location you said, it all works. :)
edit: Well the think button is there and lit up, though the model didn't actually seem to do a reasoning step on the first attempt. Will keep investigating. (Update: Should mention it works fine)