Template V10 causes hallucinations.

#10
by lorenzodimauro97 - opened

This has been tested on AesSedai Q6 and Unsloth Q8_K_XL versions of qwen 3.6 35B

When used with pi, the agent will act as if I keep repeating the same instructions.

Example of two agent turns back to back:

previous turn:

The user is showing me a long list of compilation errors from Unity. I need to fix all of them. Let me categorize them:

  1. NavMeshObstacleShape.Cylinder doesn't exist β†’ Should be NavMeshObstacleShape.Capsule (from the docs)
  2. ObstacleAvoidanceType.High doesn't exist β†’ Need to check what values are available
  3. NavMeshLinkInstance.valid is obsolete β†’ Use NavMesh.IsLinkValid() instead
  4. Transform.bounds doesn't exist β†’ Transform doesn't have bounds, need to use GetComponent().bounds
  5. HashSet.AsReadOnly() doesn't exist β†’ Need to use ToList() or return a different type
  6. DynamicObstacle._obstacle is inaccessible β†’ Need to make it internal or add a property
  7. NavMeshObstacle.Update() doesn't exist β†’ Need to check the actual API
  8. Behaviour.enabled hiding β†’ Need to rename the field or use new keyword
  9. NavMeshBaker is obsolete β†’ This is expected, we added the [Obsolete] attribute

Let me fix all these files one by one. I'll start with the most critical errors.

next turn:

The user is showing me compilation errors from Unity. I need to fix all of them. Let me categorize them:

  1. NavMeshObstacleShape.Cylinder doesn't exist β†’ Should be NavMeshObstacleShape.Capsule (from the docs I just read)
  2. ObstacleAvoidanceType.High doesn't exist β†’ Need to check what values are available
  3. NavMeshLinkInstance.valid is obsolete β†’ Use NavMesh.IsLinkValid() instead
  4. Transform.bounds doesn't exist β†’ Need to use GetComponent().bounds or similar
  5. HashSet.AsReadOnly() doesn't exist β†’ Need to use a different approach
  6. DynamicObstacle._obstacle is inaccessible β†’ Need to make it internal or add a property
  7. NavMeshObstacle.Update() doesn't exist β†’ Need to check what's available
  8. Behaviour.enabled hiding β†’ Need to rename the field or use new keyword
  9. NavMeshBaker is obsolete β†’ This is expected since we marked it as obsolete

Let me fix all these files systematically. I'll start with the most critical ones.


This is an issue because the agent will get stuck on the same task, thinking it is being reprompted for the same issues, instead of moving on / finishing.

In some cases, adding {%- set preserve_thinking = true %} to the top of the template may be beneficial. It allows the model to remember its reasoning, so perhaps it won't repeat itself with it turned on? Here's an example showing the difference:
https://www.reddit.com/r/LocalLLaMA/comments/1sne4gh/psa_qwen36_ships_with_preserve_thinking_make_sure/

Also, make sure the model isn't suspect. I was using some uncensored-wasserstein version of 35B a3b and it was terrible - switched to qwen3.6-35b-a3b-claude-4.6-opus-reasoning-distilled and the difference was night and day.

Thank you for testing and reporting. Indeed I was bit too hasty with my v10 refactoring. This should be fixed in v11 now. In my limited testing so far, so good.

For me it causes looping for some reason. Either way thank you for your work, v9 works great for now. Ill test v11 too.

Used v11 to develop a plan (VS code claude plugin, pointed to endpoints from llama-sever) - which worked fine. Implementing the plan it's getting stuck every 2 minutes, and i have to prompt it with "continue". So far >10 times. The pattern is always "Thinking...[bla bla bla]" --> "Now I need to bla bla bla" --> stuck. No such issues with v9, so going back to v9. Thanks for the great work improving the default template!

Used v11 to develop a plan (VS code claude plugin, pointed to endpoints from llama-sever) - which worked fine. Implementing the plan it's getting stuck every 2 minutes, and i have to prompt it with "continue". So far >10 times. The pattern is always "Thinking...[bla bla bla]" --> "Now I need to bla bla bla" --> stuck. No such issues with v9, so going back to v9. Thanks for the great work improving the default template!

I think this is because I mistakenly deleted the dynamic tools instruction logic and the reminder block, when I implemented the context saving compaction for tool parameters in v10. I will start working on putting it back for v12.

Sign up or log in to comment