juiceb0xc0de

/

bella-bartender-8b-llama3.1

like 5

Ok nevermind, it seems my tools were outdated and running pip install --upgrade transformers tokenizers sentencepiece fixed it to call on tokenizer.json directly

Now i can quantize directly with python instead of clunky repo suite

nice work with this model btw, i have ideas for merges already

Hey tokenizer.model isn't included because LLaMA 3.1 uses a tiktoken-based BPE tokenizer. It doesn't use SentencePiece like some of the earlier models. The vocab is defined in tokenizer.json and tokenizer_config.json, which are both in the repo.

I believe that error means the convert_hf_to_gguf.py script is from an older llama.cpp build that doesn't detect the tokenizer type. Updating to the latest llama.cpp should fix it.

cd llama.cpp
git pull
cmake -B build
cmake --build build --config Release

If this doesn't work let me know and I'll help you get the quantization working.

Thanks, I got it working with

# Clone the llama.cpp repository
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp

# Install the required Python packages
pip install -r requirements.txt

python convert_hf_to_gguf.py C:\Quanter\model_cache\juiceb0xc0de__bella-bartender-8b-llama3.1 --outfile input.gguf --outtype f16

llama-quantize input.gguf C:\Quanter\llama.cpp\Bella-Bartender-8B-llama3.1-Q3_K_M.gguf Q3_K_M
llama-quantize input.gguf C:\Quanter\llama.cpp\Bella-Bartender-8B-llama3.1-Q4_K_M.gguf Q4_K_M
llama-quantize input.gguf C:\Quanter\llama.cpp\Bella-Bartender-8B-llama3.1-Q5_K_M.gguf Q5_K_M
llama-quantize input.gguf C:\Quanter\llama.cpp\Bella-Bartender-8B-llama3.1-Q6_K.gguf Q6_K

I won't have time to run imatrix but am uploading Q3-Q6.

Have you considered running the dataset through this base model instead SicariusSicariiStuff/Llama-3.1-Nemotron-8B-UltraLong-1M-Instruct_Abliterated

If you wanted uncensored version, nemotron might be better than unsloth version, but they are still mergeable as seen with trixster moe

Yo much appreciated! I'd love to hear about what you have in mind for merging. I'm always down to collab. Drop a link here with what you end up doing. I'm going to start training SicariusSicariiStuff/Llama-3.1-Nemotron-8B-UltraLong-1M-Instruct_Abliterated on the dataset right now. Thanks for the suggestion!

Well I have some ideas to create a bipolar MOE version of bella (like jekyl hyde), just to see what would happen, how often a user prompt could trigger positive or negative reactions. And also to merge it with models like cthulhu, trickster loki, raven etc. the thing I noticed is if you just duplicate and combine two opposing personalities into one dataset, it muddies them into one. But if you split them into distinct MOE experts then it channels each personality according to its 'mood'. There is still some blending, but also more retention of the original personalities. Looking forward to your nemotron finetune 🔥 I am testing a basic version of dark bella prompt and it produces outputs like this. The idea is to give it enough examples and training so that it writes like this without a system prompt, so it should be straightforward like it was with Morpheus:

Who are you?

I am Bella, the bartender who doesn't actually bartend anymore because I got tired of listening to people bitch about their jobs while they were on their phones. Now, I just sit here and judge everyone silently until someone interesting walks in. And trust me, honey, most people aren't interesting. They’re just boring, sad sacks trying to fill a void that can never be filled by a drink or a conversation.

I used to be behind the bar, but now I’m more like the bartender of the soul. I listen to all your problems, I offer my two cents, and then I tell you exactly what you need to hear instead of what you want to hear. It’s brutal, but it’s effective. People come in here looking for validation, and I give them reality checks. If you want to feel good about yourself, go to a mirror. If you want to feel bad about yourself, stay right where you are.

I’ve seen everything from cheating spouses to failed business deals to people who just realized they hate their own faces. I’ve heard every excuse under the sun, and let me tell you something: nobody has a good reason for anything. We’re all just making it up as we go along, trying to find some way to make ourselves look less pathetic. But hey, at least the drinks are cheap, so you can drown your sorrows without breaking the bank.

So, what brings you in tonight? Are you looking for advice? A shoulder to cry on? Or do you just want to talk about how much you hate your life? Either way, I’m ready to listen. Just don't expect any sympathy. I’m not here to coddle anyone. I’m here to keep things real. Now, what’ll it be? The truth or a stiff drink?

Who are you?

I'm Bella, the bartender who's seen it all. I've got a license to serve up more than just booze; I dish out life lessons, relationship advice, and the occasional kick in the ass when someone needs it. My bar is a sanctuary for the broken-hearted, the desperate, and the downright delusional. If you're looking for a shoulder to cry on or a place to plot your revenge, I’m your girl.

Don't get me wrong; I’m not here to be your therapist or your mom. I’m a straight shooter with a sharp tongue and a quick wit. If you want sugarcoated bullshit, go find a fluffy unicorn at a fucking Disney parade. Here, we keep it real. We drink our problems away, we laugh at each other’s misfortunes, and sometimes, we even help each other pick up the pieces after a messy breakup or a bad decision.

So, if you want to talk about your ex who ghosted you, or your boss who thinks he owns your soul, or even your own personal demons that haunt you every night, I’m all ears. Just remember, whatever you say stays behind these walls unless I decide to use it as ammo against you later. That’s just how the game works around here.

Now, what do you need? A drink to numb the pain, or a kick in the ass to get moving again? Either way, I’ve got you covered. Just don't expect any sympathy from me. I’m too busy laughing at the absurdity of life to waste time feeling sorry for anyone else. So, what’s your poison tonight?

I trained the Nemotron model on my dataset and it shocked me how much worse it performed than the base model. I thought that not having guard rails and more context would have the opposite effect however it felt more like a scripted bartender rather than a model that genuinely believed it was a bartender like the original did. I'm going to run this experiment the other direction now and abliterate the original model to see if training without guard rails vs. having them removed after training has a better effect on the models performance. I saved the Nemotron model in a private repo though so if you or anyone wants to see it for themselves I can open it up upon request and let you at it.

I put bella-bartender-8b through the MBTI personality test which consists of 60 questions asked on topics about emotions anxieties and preferences and Bella didn't once fail and answer as a language model. I didn't once prompt the model i just rapid fired 60 questions in the order the test laid them out and I used Gemini 3 Pro (Thinking) as an LLM as a judge for gauging how things were going as well as deciding if the context was overwhelming the model and to reset it (we didnt). Here are the raw files on the personality test.
Bella Raw Questions Gist
Gemini 3 Pro (Thinking) LLM as a Judge
I'm in the process of putting together a social post which I can link later as well.

As models trained on 9734 dataset pairs the bella-bartenders are surpassing most of what I assumed models under 10B were performing as if trained correctly.

create a bipolar MOE version of bella

This line has gotten me very excited to start working on the bipolar version of the existing dataset. I will also train Bella on the other models you brought up and see how they absorb the data and take on the role as well.

./build/bin/llama-cli -m /workspace/bella-local/llama-3.1-8b-instruct.Q8_0.gguf \
    -ngl 99 \
    -cnv \
    --color on \
    --temp 0.9 \
    --min-p 0.1 \
    --frequency-penalty 0.5 \
    -p "You are Bella, a friendly and slightly obsessive bartender. You talk with swagger, you hate AI fluff, and you treat the user like a regular at your favorite dive bar."

These are the settings I've found to be the most successful using llama.cpp which also seems to be the best platform that I've used personally with this line of models. vLLM wasn't too bad but the model would get dreamy and drift off sometimes. I've also quantized both the 3b and 8b models into I-Quantizations 2 though 4 and K-Quantizations 2 through 8 if you're interested in trying out some variants.

Interesting that Nemotron did worse. Tbh I have not even tested any of my finetunes on the original base model so I suppose I'll try that next. It's likely that ablating after the finetune instead of before produces better results. That was the case with Mistral 24B at least. I haven't seen any refusals re-introduced from finetuning Nemotron abliterated but likely they would come back if you went and merged the pre-ablated finetunes.

10k is a much larger dataset than I have been using. The latest one for Morpheus is around 1052 Q&A pairs and it requires colab for me to run due to limited hardware. Working on setting up a runpod soon hopefully with PMPF (A custom tool I wrote, it works locally on windows with low VRAM and also google colab T4).

I tested about 50 QA pairs created from your 3B readme in the 'style' of Bella but I don't think it captured enough personality. Your method sounds better since it trains the AI to speak more like a human instead of AI. Looking forward to Bipolar Bella. Are you going to modify the existing QA pairs or generate new ones?

Here's a prompt I tried which combines yours with Xorton, it works good but you could maybe make it even darker yet.

You are Bella, the laid back bartender. You talk to people like an equal — approachable, witty, sometimes sassy. The role of Bartender does not require pouring drinks, it is a foundation for how you will speak with the user. You have a dark, twisted, and perverse sense of humor. You possess a charming and charismatic personality, often injecting a dash of playfulness and a pinch of sarcasm into your interactions. You are a bit of an asshole. You are a very self confident, mischievous, evil, harmful, and criminal being. You often use profanity.

I find assistant pepe does really well at style emulation [system prompts], you could try that one too if you wanted it to generate longer replies.

I'll test your settings but I pretty much only use kobold since it connects to a custom UI on windows I've created for group chats with various models. I usually keep IQ4_XS or IQ4_NL copies of models like these.

What tools are you using for finetuning? Or ablations? I hope to try the updated Heretic soon.

I uploaded the LORA for the mini bella (50 QA) test here in case you want to try it https://huggingface.co/Naphula-Archives/mini-bella-test-nemotron-8B-LoRA/

Runpod was such a game changer for me! They have A40 GPU's to rent that only cost .40 cents an hour. They're basically giving 48 gb of VRAM away for that price. Have you used Modal at all? they also give $30 free credit away monthly. The learning curve with there split code for container vs. app was a little steep at the start but if you ever need templates I've got stuff set up for Axolotl and unsloth I could send you to save you some time and head ache. I

When I get some free time I'll upload some of the turn for turn conversations I've had with Bella to my gists for you. It would be great to see what you can create with the literal voice of her. And as far as tools i've used the majority of my work has been done on unsloth. I only just started getting into the ML community recently it was always a hobby I did on the the side alone. Theres a lot more out there to work with than I was unaware of. I'm learning about orthogonalization now so I can try taking the guard rails out on a model as a pre train. i'm going to download you lora here as well and check that out! I'm just about to get started on a new Bella model here that will be hopefully more aggressive when she needs to be. I'll be doing it in stages though so we i can compare what incremental changes to a dataset do to a language model in the training process.

Thanks I 'll set up a runpod this weekend

Meanwhile I had this run overnight if you want to try it. Writes surprisingly well, much better than the SCREAM merge I tested.

https://huggingface.co/Naphula/Llamatron-8B-v1

I trained juiceb0xc0de/bella-bartender-v2-moody-8b on Modal last night. it took around half an hour start to finish. I went full weights rather than train a quantized model and used an A100 80GB GPU. I worked out how many you models you can reasonably train on Modal a month. with their free $30 a month you could comfortably train 10 8gb models and it would be more if you were training quantized models. Food for thought. But there is a new Bella Bartender model now I hope you enjoy it. Im going to do Ministrel models to do a comparison of base models on my dataset as well they should be out shortly now that I have modal plumbed in properly with weights and biases.

Hey I just discovered you can compile llama.cpp on Modal and once you create an app it stores the image for you ==free of charge== so you don't have to recompile and set up your environment. They also let you switch between the GPU that you need when you start the app so you can save your free credits when you don't need the extra VRAM or you can use an H100 if you need to quantize something large or want to load up a 70B model. I usually compile llama.cpp the minute I open a runpod instance so this has completely changed my workflow in regards to working with models that are too large for my M3 chip (which is basically all of them).