KenSensei/gemma-3-270m-openthoughts

🤖 Gemma-3-270M Fine-Tuned on OpenThoughts-114k

A compact, reasoning-capable assistant trained on 114K high-quality synthetic examples covering math, science, code, and logic puzzles.

---

🌟 Overview

This model is a LoRA fine-tuned version of gemma-3-270m-it, trained on the OpenThoughts-114k dataset — a high-quality synthetic reasoning dataset with 114,000 examples spanning math, coding, science, and puzzles.

✅ Trained with Unsloth for 2× faster training & 60% less VRAM
✅ Supports local inference via GGUF (Q8_0, BF16) and continued training via LoRA
✅ Ideal for local LLM enthusiasts, developers, and researchers needing a small but smart model

---

📊 Model Details

Attribute	Value
Base Model	unsloth/gemma-3-270m-it
Dataset	open-thoughts/OpenThoughts-114k
Training Method	LoRA (rank=64) + Unsloth-optimized SFT
Context Length	2048 tokens
Hardware Used	Google Colab T4 (16GB VRAM)
Training Steps	300 steps (~30 minutes)
Final Loss	~2.81
Trainable Params	15.2M (5.36% of total)
License	LGPL-3.0

🔍 The dataset uses a structured "Thought → Solution" format, teaching the model to reason step-by-step before answering — ideal for complex tasks.

---

🚀 Quick Start

▶️ Run with Ollama

# Create Modelfile
echo 'FROM ./gemma-3-270m-openthoughts.Q8_0.gguf
TEMPLATE "{{ if .Prompt }}<start_of_turn>user\n{{ .Prompt }}<end_of_turn>\n<start_of_turn>model\n{{ end }}"
PARAMETER temperature 0.8
PARAMETER top_p 0.95
PARAMETER top_k 64' > Modelfile

# Build and run
ollama create my-gemma -f Modelfile
ollama run my-gemma

▶️ Use with llama.cpp

./main -m gemma-3-270m-openthoughts.Q8_0.gguf -p "Explain backpropagation like I'm 12." -n 150

▶️ Load in Python (Transformers + Unsloth)

from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained("your-hf-username/gemma-3-270m-openthoughts")
# Use for inference or continued training

---

📥 Download Files

File	Format	Use Case
gemma-3-270m-openthoughts.Q8_0.gguf	8-bit quantized	Best for llama.cpp, Ollama, LM Studio
gemma-3-270m-openthoughts.BF16.gguf	Full precision	Highest quality, larger size
adapter_model.safetensors	LoRA adapter	For continued training or merging

💡 Recommendation: Use Q8_0 for most local applications (~500 MB). Use BF16 only if you need maximum fidelity.

---

🔁 Continue Training

You can resume training on new datasets (e.g., teknium/OpenHermes-2.5) using the LoRA checkpoint:

from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
    model_name = "your-hf-username/gemma-3-270m-openthoughts",  # LoRA folder
    max_seq_length = 2048,
    load_in_4bit = True,
)
# Add new data and train!

---

🧠 Why OpenThoughts?

• ✅ 114K diverse reasoning examples
• ✅ Verified correctness (solutions validated during generation)
• ✅ Covers math, algorithms, physics, logic puzzles
• ✅ Structured chain-of-thought format improves reasoning
• ✅ Fully open-source (paper, code, data available)

📄 Read the OpenThoughts paper for full methodology.

---

🛠️ Training Code (Unsloth)

Trained in <30 minutes on a free T4 using:

from unsloth import FastModel
from datasets import load_dataset

# Load model
model, tokenizer = FastModel.from_pretrained(
    "unsloth/gemma-3-270m-it",
    load_in_4bit=True,
    max_seq_length=2048,
)

# Apply LoRA
model = FastModel.get_peft_model(model, r=64)

# Load & format OpenThoughts
dataset = load_dataset("open-thoughts/OpenThoughts-114k", split="train[:50000]")
# (system + conversations → user/assistant only for Gemma-3)

# Train
trainer = SFTTrainer(..., max_steps=300, per_device_train_batch_size=4)
trainer.train()

# Export to GGUF
model.save_pretrained_gguf("gemma-3-270m-openthoughts", tokenizer, quantization_method="Q8_0")

---

📜 License

• Model: LGPL-3.0 (Unsloth License)
• Base Gemma-3: Google Gemma Terms
• Dataset: OpenThoughts License

⚠️ Commercial use: Review Gemma’s terms before deployment.

---

🙌 Acknowledgements

• Unsloth — for blazing-fast, memory-efficient fine-tuning
• OpenThoughts Team — for the high-quality reasoning dataset
• Hugging Face — for the ecosystem and tools