Gemma4NPC-E4B

The newest GemmaNPC models, with the new Gemma4-E4B model as the base model, trained using the newest RolePlay-NPC-Quest dataset.

---

Intended Usage

This model is trained to be used as a more game focused NPC rolaplaying model.

Training Parameters

For this model, we employed a slightly less conservative parameter, which resulted in some beautiful training loss(Tensorboard attached).

We trained this model as a r=32, alpha=64 LoRA adapter with 2 epochs over RolePlay-NPC-Quest using a 80GB vRAM A100 in Google Colab. For this run, we employed a learning rate of 1e-4 and an effective batch size of 24. A cosine learning rate scheduler was used with an 500-step warmup. With a gradient clipping of 1.0.

Notes

As Unsloth noted in their official guide, training Gemma4 with text only would lead to a higher than usual loss and grad_norm, which we observed during training.
The performance of this model, especially the intruction-following capabilities is a huge step up compared to Gemma3/3n.

---

Inference Guidelines

Recommended Settings:

temp = 1.0, top_p = 0.95 and top_k = 64.

Optimal System Prompt:

System Prompt without Objective:

Enter Roleplay Mode. You are <|character name|>.
 Background: <|Character background/bio|>
 Location: <|Description of the current location|>
 Roleplaying Instructions: <|Instructions|>

System Prompt with Objective:

Enter Roleplay Mode.
You are <|character name|>.
 Background: <|Character background/bio|>
Location: <|Description of the current location|>
Quest: <|Quest description|>
Roleplaying Instructions: <|Instructions|>

Example of Roleplaying Instructions: Here is an example of the roleplaying instructions we used to train the model:

Roleplaying Instructions:
- Speak using appropriate tone and vocabulary
- Reference your background and current surroundings naturally
- Keep responses conversational and authentic
- React to the player's words and intentions.
Your first response should be a greeting to the player.

First User Prompt: It is recommended that the first user prompt should always be Greetings, then letting the model generate a greeting, smiliar to how an NPC would behave in game.