Thank You for the Detailed Description
I'm always happy to see the creator of a model provide the settings for it. It was so easy to import the master settings in SillyTavern. Even better, you added extensive and detailed info on your thoughts on the model, about the training, the model's capabilities, etc. It was a great read. Thank you!
The model itself is fantastic for a 12B LLM. I can't even fathom how long it took you to train it on 1 BILLION+ tokens. With what little testing I've done, the Q8 fits great in my 16GB VRAM GPU, and runs blazingly fast at Q8 KV Cache via Ollama. And it seems way smarter as well. I tested it with my own char card and it adapted to it perfectly without additional instructions. I'm simply amazed.
If you don't mind, can you tell me what system prompt you use for your roleplay cards? Just the default SillyTavern one or something else?
And is the max context length the same as for the OG model, 128k or is it capped at 16k as per the master settings?
Also, one last question, how do you enable this orange bar next to the user's message and what does it do compared to just typing out the message and sending it:
Thanks again for this fun model, it has become my go-to because of how fast it is and how intelligent its responses are...
Happy that you're happy :)
Regarding context, from manual test up to 18k tokens, it hold up well. Beyond that I haven't tested \ gotten any test reports, but... it is likely that ~20k tokens are the limit.
Of course, this could be somewhat artificially extended with various long term memory plugins etc etc...
For your other question, the system prompt should be empty, and use only the character card.
The Fallout New-Reno card uses the following (you can adapt this style for any adventure scenario, very simple, flexible and token efficient):
Adventure: {{user}} is a male lone wanderer in the post-nuclear wasteland, 200 years after the Great War. {{user}}'s conditions and items are tracked with precision.
$Stats: High Endurance, Moderate Strength, Low Intelligence, Very Low Charisma, High Luck, Moderate Perception, Low Agility
$Traits: Bloody Mess - Violence around {{user}} becomes extraordinarily gory and destructive. Jinxed - Mechanical devices frequently malfunction in {{user}}'s presence, affecting both enemies and {{user}}.
$Starting Equipment: Tattered vault suit (no armor value), empty 10mm pistol, 69 bottle caps, 12 Crayons, rusty knife
$Current Condition: Lame leg due to injury
$World_Setting: Fallout universe, The Capital Wasteland - a hellscape of raiders, super mutants, feral ghouls, and worse.
Important regarding adventure mode:
the reply format works best when started with > following by first person action, and if needed "speech"
Example:
>I grab the Limeware Platter in front of him and say: "Look behind you!"
Awesome, thank you for getting back to me! I'll keep your info in mind, especially with the context, and the sys prompt needing to be empty. But I am using your character cards for roleplaying, and so far Adventure Morrowind - Male Bosmer is going really well. Nikki card is also great, something fresh and neat! I'll check out your Fallout New-Reno card as well, and see how far I can go even tho I never played the games set in the universe. Thanks again for all your work, these 12b models pack a punch for sure!