The model seems very dumb. Am I using it wrong?
#4
by unbalancedmercy - opened
I am using TheDrummer_Snowpiercer-15B-v3-Q4_K_S.gguf. I'm running the model in KoboldCPP and connecting with SillyTavern and the ChatML context and instruct templates. My sampler settings are:
Temperature = 0.6
Top nsigma = 1.5
XTC threshold = 0.1
XTC probability = 0.2
DRY multiplier = 0.85
DRY base = 1.75
Meanwhile, I'm commonly getting junk output like "She closes her eyes, leaning her head back against the tiled." where that sentence should probably end with "tiles." or "tiled wall." I'm surprised to see that with my temperature so low. Does anyone know how to fix this?