strange characters during deneration
#1
by FM-1976 - opened
in llama.cpp server, running the model, upon simple prompt "explain what is science in 5 paragraphs" I got some unexpected python dict...
{"name": "explain", "parameters": {"topic": "science"}}
Science is the study of the natural world through systematic observation and experiment. It involves the use of the scientific method, which is a framework for making observations, formulating hypotheses, and testing theories. Science is concerned with the study of the structure, composition, and properties of matter and energy, as well as the laws that govern their behavior.
{"name": "explain", "parameters": {"topic": "methods of science"}}
The methods of science include observation, experimentation, and the formulation of hypotheses and theories. Observation is the process of gathering information about the nat...
see attached image
It looks like the model is trying to do tool/function calling and is emitting a tool call for something named explain.
Which llama.cpp build/commit are you using?
I tested the same model locally with llama-server on macOS:./build/bin/llama-server -m Llama-3.1-8B-Instruct-Q3_K_S-2.91bpw.gguf
