Upload EthicalHacking_Gemma4_E2B_Colab.ipynb

EthicalHacking_Gemma4_E2B_Colab.ipynb

@@ -4,27 +4,26 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "# 🔐 Ultimate Ethical Hacking LLM – Gemma 4 E2B (Colab Free Tier T4)\n",
+    "# 🔐 Ultimate LLM Fine-Tuning – Gemma 4 E2B (Colab Free Tier T4)\n",
     "\n",
     "**🥇 Model:** [Google Gemma 4 E2B](https://huggingface.co/google/gemma-4-E2B-it) via Unsloth 4-bit  \n",
-    "**🏆 Why this model?** Dense ~2B parameter edge model. NOT an MoE — all 2B params are active every forward pass. Strong reasoning for its size.  \n",
-    "**⚠️ T4 WARNING:** This is **tight on 16GB VRAM**. The 4-bit model alone uses ~7.4GB. You MUST follow the memory-optimized settings below.  \n",
-    "**📊 Datasets:** [Fenrir v2.1](https://huggingface.co/datasets/AlicanKiraz0/Cybersecurity-Dataset-Fenrir-v2.1) + [Trendyol Cybersecurity](https://huggingface.co/datasets/Trendyol/Trendyol-Cybersecurity-Instruction-Tuning-Dataset)  \n",
+    "**🏆 Why this model?** Dense ~2B parameter edge model. NOT an MoE — all 2B params are active.  \n",
+    "**⚠️ T4 WARNING:** This is **tight on 16GB VRAM**. The 4-bit model uses ~7.4GB. Follow memory settings strictly.  \n",
+    "**📊 Datasets:** Your choice — cybersecurity, general chat, multilingual, coding, or mix them!  \n",
     "**⚡ Framework:** Unsloth + TRL SFTTrainer  \n",
     "\n",
-    "> ⚠️ **Disclaimer:** Defensive cybersecurity datasets only. Ethical hacking education.\n",
+    "> Pick any dataset below. Default is cybersecurity. Mix datasets for hybrid tuning.\n",
     "\n",
     "---\n",
     "\n",
-    "## 📋 Why Gemma-4 E2B?\n",
+    "## 📋 Gemma-4 E2B T4 Notes\n",
     "\n",
     "| Spec | Value |\n",
     "|------|-------|\n",
     "| Parameters | ~2B (dense, NOT MoE) |\n",
     "| 4-bit VRAM | ~7.4 GB |\n",
-    "| Context | Up to 256K tokens |\n",
     "| Batch size on T4 | **1 only** |\n",
-    "| Max seq length | **2048 max** on T4 |\n",
+    "| Max seq length | **2048 max** |\n",
     "| LoRA rank | **8** (save VRAM) |\n",
     "\n",
     "**Unsloth docs:** https://unsloth.ai/docs/models/gemma-4/train  \n",
@@ -71,15 +70,15 @@
    "source": [
     "## 3️⃣ Load Gemma-4 E2B in 4-bit via Unsloth\n",
     "\n",
-    "**⚠️ T4 MEMORY LIMITS — READ CAREFULLY:**\n",
+    "**⚠️ MEMORY LIMITS:**\n",
     "\n",
     "| Setting | Value | Why |\n",
     "|---------|-------|-----|\n",
     "| `BATCH_SIZE` | **1** | Cannot fit >1 on T4 |\n",
-    "| `MAX_SEQ_LENGTH` | **2048** | Longer = OOM during backprop |\n",
-    "| `LORA_R` | **8** | Small rank = fewer adapter params |\n",
-    "| `GRAD_ACCUM` | **8** | Effective batch still = 8 |\n",
-    "| `PACKING` | **False** | Avoids complex memory spikes |\n",
+    "| `MAX_SEQ_LENGTH` | **2048** | Longer = OOM |\n",
+    "| `LORA_R` | **8** | Small rank saves VRAM |\n",
+    "| `GRAD_ACCUM` | **8** | Effective batch = 8 |\n",
+    "| `PACKING` | **False** | Safer memory profile |\n",
     "| `optim` | `adamw_8bit` | Must use 8-bit optimizer |\n",
     "\n",
     "If you still OOM: lower `MAX_SEQ_LENGTH` to 1024, or use `use_rslora=True`."
@@ -98,27 +97,24 @@
     "MAX_SEQ_LENGTH = 2048          # DO NOT exceed 2048 on T4\n",
     "LORA_R = 8                     # small rank for memory\n",
     "LORA_ALPHA = 8                 \n",
-    "BATCH_SIZE = 1                 # MUST be 1 on T4 (model is ~7.4GB in 4-bit)\n",
+    "BATCH_SIZE = 1                 # MUST be 1 on T4\n",
     "GRAD_ACCUM = 8                 # effective batch = 8\n",
     "LEARNING_RATE = 2e-4           \n",
-    "NUM_EPOCHS = 1\n",
     "MAX_STEPS = 4000               \n",
-    "WARMUP_STEPS = 100             # shorter warmup (tight memory)\n",
+    "WARMUP_STEPS = 100             \n",
     "LOGGING_STEPS = 50             \n",
     "SAVE_STEPS = 500               \n",
-    "PACKING = False                # False = simpler memory profile\n",
+    "PACKING = False                # False = safer memory\n",
     "SAMPLE_SIZE = 50000            \n",
-    "HUB_MODEL_ID = \"your-username/cyber-gemma4-e2b-lora\"  \n",
+    "HUB_MODEL_ID = \"your-username/gemma4-e2b-lora\"\n",
     "# ================================================================================\n",
     "\n",
-    "# NOTE: Unsloth auto-applies 4-bit when loading Gemma-4.\n",
-    "# If the unsloth-bnb-4bit ID doesn't exist, try the base unsloth ID with load_in_4bit=True.\n",
     "MODEL_NAME = \"unsloth/gemma-4-E2B-it-unsloth-bnb-4bit\"  # ~7.6GB download\n",
     "\n",
     "model, tokenizer = FastLanguageModel.from_pretrained(\n",
     "    model_name=MODEL_NAME,\n",
     "    max_seq_length=MAX_SEQ_LENGTH,\n",
-    "    dtype=None,                   # auto-detect (fp16 on T4)\n",
+    "    dtype=None,\n",
     "    load_in_4bit=True,\n",
     ")\n",
     "\n",
@@ -128,18 +124,18 @@
     "    target_modules=[\"q_proj\", \"k_proj\", \"v_proj\", \"o_proj\",\n",
     "                   \"gate_proj\", \"up_proj\", \"down_proj\"],\n",
     "    lora_alpha=LORA_ALPHA,\n",
-    "    lora_dropout=0,               \n",
+    "    lora_dropout=0,\n",
     "    bias=\"none\",\n",
     "    use_gradient_checkpointing=\"unsloth\",  # CRITICAL for T4\n",
     "    random_state=3407,\n",
-    "    use_rslora=False,             # set True if still OOM\n",
+    "    use_rslora=False,\n",
     "    loftq_config=None,\n",
     ")\n",
     "\n",
     "trainable = sum(p.numel() for p in model.parameters() if p.requires_grad)\n",
     "total     = sum(p.numel() for p in model.parameters())\n",
     "print(f\"✅ Gemma-4 E2B loaded. Trainable params: {trainable:,} / {total:,} ({100*trainable/total:.2f}%)\")\n",
-    "print(f\"⚠️  This model is LARGE. Expected VRAM during training: ~12-14 GB\")\n",
+    "print(f\"⚠️  Expected training VRAM: ~12-14 GB (out of 16 GB)\")\n",
     "print(f\"    If you get OOM, lower MAX_SEQ_LENGTH to 1024 or set use_rslora=True\")"
    ]
   },
@@ -147,7 +143,19 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## 4️⃣ Load, Audit, Subsample & Merge Cybersecurity Datasets"
+    "## 4️⃣ 🎯 CHOOSE YOUR DATASET(S)\n",
+    "\n",
+    "Uncomment **ONE** `DATASET_CHOICE` line. Mix datasets with `custom_mix`.\n",
+    "\n",
+    "| Choice | Dataset | Size | Format | Best For |\n",
+    "|--------|---------|------|--------|----------|\n",
+    "| `\"cybersecurity\"` | Fenrir + Trendyol | 153K | system/user/assistant | **Ethical hacking education** |\n",
+    "| `\"ultrachat\"` | UltraChat 200K SFT | 200K | messages | General conversation |\n",
+    "| `\"openhermes\"` | OpenHermes 2.5 | 1M+ | conversations | Reasoning, coding |\n",
+    "| `\"sharegpt_en\"` | ShareGPT English | ~90K | conversations | Multi-turn dialogue |\n",
+    "| `\"sharegpt_de\"` | ShareGPT German | ~104K | conversations | German fine-tuning |\n",
+    "| `\"sharegpt_hi\"` | ShareGPT Hindi | ~153K | conversations | Hindi fine-tuning |\n",
+    "| `\"custom_mix\"` | Your mix | — | varies | Combine multiple |"
    ]
   },
   {
@@ -157,53 +165,119 @@
    "outputs": [],
    "source": [
     "from datasets import load_dataset, concatenate_datasets\n",
+    "\n",
+    "# ═══════════════════════════════════════════════════════════════\n",
+    "#   SELECT YOUR DATASET — UNCOMMENT ONE LINE\n",
+    "# ═══════════════════════════════════════════════════════════════\n",
+    "\n",
+    "DATASET_CHOICE = \"cybersecurity\"\n",
+    "\n",
+    "# DATASET_CHOICE = \"ultrachat\"\n",
+    "# DATASET_CHOICE = \"openhermes\"\n",
+    "# DATASET_CHOICE = \"sharegpt_en\"\n",
+    "# DATASET_CHOICE = \"sharegpt_de\"\n",
+    "# DATASET_CHOICE = \"sharegpt_hi\"\n",
+    "# DATASET_CHOICE = \"custom_mix\"\n",
+    "\n",
+    "CUSTOM_DATASETS = [\n",
+    "    (\"AlicanKiraz0/Cybersecurity-Dataset-Fenrir-v2.1\", \"train\", 10000, \"messages\"),\n",
+    "    (\"HuggingFaceH4/ultrachat_200k\", \"train_sft\", 20000, \"messages\"),\n",
+    "    (\"teknium/OpenHermes-2.5\", \"train\", 20000, \"conversations\"),\n",
+    "]\n",
+    "\n",
+    "print(f\"🎯 DATASET_CHOICE = {DATASET_CHOICE}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 5️⃣ Load, Convert & Pre-process Selected Dataset"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
     "import random\n",
     "\n",
-    "# ---------- Dataset 1: Fenrir v2.1 ----------\n",
-    "print(\"📥 Loading Fenrir v2.1...\")\n",
-    "ds1 = load_dataset(\"AlicanKiraz0/Cybersecurity-Dataset-Fenrir-v2.1\", split=\"train\")\n",
-    "print(f\"   Rows: {len(ds1)} | Columns: {ds1.column_names}\")\n",
-    "\n",
-    "for i in random.sample(range(len(ds1)), 2):\n",
-    "    print(f\"\\n--- Sample {i} ---\")\n",
-    "    print(f\"SYSTEM: {ds1[i]['system'][:120]}...\")\n",
-    "    print(f\"USER:   {ds1[i]['user'][:120]}...\")\n",
-    "    print(f\"ASSIST: {ds1[i]['assistant'][:120]}...\")\n",
-    "\n",
-    "def fenrir_to_messages(example):\n",
-    "    return {\n",
-    "        \"messages\": [\n",
-    "            {\"role\": \"system\",    \"content\": example[\"system\"]},\n",
-    "            {\"role\": \"user\",      \"content\": example[\"user\"]},\n",
-    "            {\"role\": \"assistant\", \"content\": example[\"assistant\"]},\n",
-    "        ]\n",
-    "    }\n",
-    "\n",
-    "ds1 = ds1.map(fenrir_to_messages, remove_columns=ds1.column_names, batched=False)\n",
-    "\n",
-    "# ---------- Dataset 2: Trendyol ----------\n",
-    "print(\"\\n📥 Loading Trendyol Cybersecurity...\")\n",
-    "ds2 = load_dataset(\"Trendyol/Trendyol-Cybersecurity-Instruction-Tuning-Dataset\", split=\"train\")\n",
-    "print(f\"   Rows: {len(ds2)} | Columns: {ds2.column_names}\")\n",
-    "\n",
-    "def trendyol_to_messages(example):\n",
-    "    return {\n",
-    "        \"messages\": [\n",
-    "            {\"role\": \"system\",    \"content\": example[\"system\"]},\n",
-    "            {\"role\": \"user\",      \"content\": example[\"user\"]},\n",
-    "            {\"role\": \"assistant\", \"content\": example[\"assistant\"]},\n",
-    "        ]\n",
-    "    }\n",
-    "\n",
-    "ds2 = ds2.map(trendyol_to_messages, remove_columns=ds2.column_names, batched=False)\n",
-    "\n",
-    "# ---------- Merge & Subsample ----------\n",
-    "train_dataset = concatenate_datasets([ds1, ds2])\n",
+    "def _convert_fenrir(example):\n",
+    "    return {\"messages\": [\n",
+    "        {\"role\": \"system\",    \"content\": example[\"system\"]},\n",
+    "        {\"role\": \"user\",      \"content\": example[\"user\"]},\n",
+    "        {\"role\": \"assistant\", \"content\": example[\"assistant\"]},\n",
+    "    ]}\n",
+    "\n",
+    "def _convert_trendyol(example):\n",
+    "    return {\"messages\": [\n",
+    "        {\"role\": \"system\",    \"content\": example[\"system\"]},\n",
+    "        {\"role\": \"user\",      \"content\": example[\"user\"]},\n",
+    "        {\"role\": \"assistant\", \"content\": example[\"assistant\"]},\n",
+    "    ]}\n",
+    "\n",
+    "def _convert_ultrachat(example):\n",
+    "    return {\"messages\": example[\"messages\"]}\n",
+    "\n",
+    "def _convert_conversations(example):\n",
+    "    msgs = []\n",
+    "    system = example.get(\"system_prompt\", \"\") or example.get(\"system\", \"\")\n",
+    "    if system:\n",
+    "        msgs.append({\"role\": \"system\", \"content\": system})\n",
+    "    for turn in example[\"conversations\"]:\n",
+    "        role = \"user\" if turn[\"from\"] in (\"human\", \"user\") else \"assistant\"\n",
+    "        msgs.append({\"role\": role, \"content\": turn[\"value\"]})\n",
+    "    return {\"messages\": msgs}\n",
+    "\n",
+    "all_datasets = []\n",
+    "\n",
+    "if DATASET_CHOICE == \"cybersecurity\":\n",
+    "    ds1 = load_dataset(\"AlicanKiraz0/Cybersecurity-Dataset-Fenrir-v2.1\", split=\"train\")\n",
+    "    ds1 = ds1.map(_convert_fenrir, remove_columns=ds1.column_names, batched=False)\n",
+    "    all_datasets.append(ds1)\n",
+    "    ds2 = load_dataset(\"Trendyol/Trendyol-Cybersecurity-Instruction-Tuning-Dataset\", split=\"train\")\n",
+    "    ds2 = ds2.map(_convert_trendyol, remove_columns=ds2.column_names, batched=False)\n",
+    "    all_datasets.append(ds2)\n",
+    "\n",
+    "elif DATASET_CHOICE == \"ultrachat\":\n",
+    "    ds = load_dataset(\"HuggingFaceH4/ultrachat_200k\", split=\"train_sft\")\n",
+    "    ds = ds.map(_convert_ultrachat, remove_columns=ds.column_names, batched=False)\n",
+    "    all_datasets.append(ds)\n",
+    "\n",
+    "elif DATASET_CHOICE == \"openhermes\":\n",
+    "    ds = load_dataset(\"teknium/OpenHermes-2.5\", split=\"train\")\n",
+    "    ds = ds.map(_convert_conversations, remove_columns=ds.column_names, batched=False)\n",
+    "    all_datasets.append(ds)\n",
+    "\n",
+    "elif DATASET_CHOICE.startswith(\"sharegpt_\"):\n",
+    "    split_map = {\"sharegpt_en\": \"english\", \"sharegpt_de\": \"german_4b_translated\", \"sharegpt_hi\": \"hindi_27b_translated\"}\n",
+    "    ds = load_dataset(\"deepmage121/ShareGPT_multilingual\", split=split_map[DATASET_CHOICE])\n",
+    "    ds = ds.map(_convert_conversations, remove_columns=ds.column_names, batched=False)\n",
+    "    all_datasets.append(ds)\n",
+    "\n",
+    "elif DATASET_CHOICE == \"custom_mix\":\n",
+    "    for ds_id, split, n_rows, fmt in CUSTOM_DATASETS:\n",
+    "        ds = load_dataset(ds_id, split=split)\n",
+    "        if n_rows and len(ds) > n_rows:\n",
+    "            ds = ds.shuffle(seed=3407).select(range(n_rows))\n",
+    "        if fmt == \"messages\": ds = ds.map(_convert_ultrachat, remove_columns=ds.column_names, batched=False)\n",
+    "        elif fmt == \"conversations\": ds = ds.map(_convert_conversations, remove_columns=ds.column_names, batched=False)\n",
+    "        all_datasets.append(ds)\n",
+    "\n",
+    "else:\n",
+    "    raise ValueError(f\"Unknown DATASET_CHOICE: {DATASET_CHOICE}\")\n",
+    "\n",
+    "train_dataset = concatenate_datasets(all_datasets) if len(all_datasets) > 1 else all_datasets[0]\n",
     "print(f\"\\n📊 COMBINED DATASET: {len(train_dataset)} rows\")\n",
     "\n",
+    "sample = train_dataset[random.randint(0, len(train_dataset)-1)]\n",
+    "print(f\"Sample roles: {[m['role'] for m in sample['messages']]}\")\n",
+    "for m in sample[\"messages\"]: print(f\"  {m['role']}: {m['content'][:80]}...\")\n",
+    "\n",
     "if len(train_dataset) > SAMPLE_SIZE:\n",
     "    train_dataset = train_dataset.shuffle(seed=3407).select(range(SAMPLE_SIZE))\n",
-    "    print(f\"🚀 SUBSAMPLED to {len(train_dataset)} rows\")\n",
+    "    print(f\"\\n🚀 SUBSAMPLED to {len(train_dataset)} rows\")\n",
     "\n",
     "print(f\"   Effective batch size: {BATCH_SIZE * GRAD_ACCUM}\")\n",
     "print(f\"   Steps per epoch: ~{len(train_dataset) // (BATCH_SIZE * GRAD_ACCUM)}\")\n",
@@ -214,7 +288,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## 5️⃣ Pre-process Dataset to Text (Avoid Unsloth formatting_func issues)"
+    "## 6️⃣ Convert Messages → Text (Chat Template)"
    ]
   },
   {
@@ -226,22 +300,12 @@
     "def convert_messages_to_text(examples):\n",
     "    texts = []\n",
     "    for msgs in examples[\"messages\"]:\n",
-    "        text = tokenizer.apply_chat_template(\n",
-    "            msgs,\n",
-    "            tokenize=False,\n",
-    "            add_generation_prompt=False,\n",
-    "        )\n",
+    "        text = tokenizer.apply_chat_template(msgs, tokenize=False, add_generation_prompt=False)\n",
     "        texts.append(text)\n",
     "    return {\"text\": texts}\n",
     "\n",
     "print(\"🔄 Converting messages to text...\")\n",
-    "train_dataset = train_dataset.map(\n",
-    "    convert_messages_to_text,\n",
-    "    batched=True,\n",
-    "    remove_columns=[\"messages\"],\n",
-    "    batch_size=100,\n",
-    ")\n",
-    "\n",
+    "train_dataset = train_dataset.map(convert_messages_to_text, batched=True, remove_columns=[\"messages\"], batch_size=100)\n",
     "print(f\"✅ Dataset pre-processed. Columns: {train_dataset.column_names}\")\n",
     "print(f\"📄 Sample text length: {len(train_dataset[0]['text'])} chars\")"
    ]
@@ -250,7 +314,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## 6️⃣ Configure SFT Trainer (T4-Safe Memory Settings)"
+    "## 7️⃣ Configure SFT Trainer (T4-Safe Memory Settings)"
    ]
   },
   {
@@ -269,16 +333,16 @@
     "    dataset_text_field=\"text\",\n",
     "    max_seq_length=MAX_SEQ_LENGTH,\n",
     "    dataset_num_proc=2,\n",
-    "    packing=PACKING,                # False = safer for T4 with large model\n",
+    "    packing=PACKING,\n",
     "    args=TrainingArguments(\n",
-    "        per_device_train_batch_size=BATCH_SIZE,     # MUST be 1\n",
-    "        gradient_accumulation_steps=GRAD_ACCUM,     # effective batch = 8\n",
+    "        per_device_train_batch_size=BATCH_SIZE,\n",
+    "        gradient_accumulation_steps=GRAD_ACCUM,\n",
     "        warmup_steps=WARMUP_STEPS,\n",
     "        max_steps=MAX_STEPS,\n",
     "        learning_rate=LEARNING_RATE,\n",
-    "        fp16=True,                      # T4 = fp16 only\n",
+    "        fp16=True,\n",
     "        logging_steps=LOGGING_STEPS,\n",
-    "        optim=\"adamw_8bit\",         # CRITICAL: saves ~2-3GB VRAM\n",
+    "        optim=\"adamw_8bit\",\n",
     "        weight_decay=0.01,\n",
     "        lr_scheduler_type=\"linear\",\n",
     "        seed=3407,\n",
@@ -287,11 +351,10 @@
     "        save_steps=SAVE_STEPS,\n",
     "        save_total_limit=2,\n",
     "        report_to=\"none\",\n",
-    "        # gradient_checkpointing=True,    # already set via use_gradient_checkpointing in LoRA\n",
     "    ),\n",
     ")\n",
     "\n",
-    "print(f\"✅ Trainer ready. Total steps: {MAX_STEPS}\")\n",
+    "print(f\"✅ Trainer ready. Dataset: {DATASET_CHOICE} | Steps: {MAX_STEPS}\")\n",
     "print(f\"   Effective batch size: {BATCH_SIZE * GRAD_ACCUM}\")\n",
     "print(f\"   Packing enabled: {PACKING}\")\n",
     "print(f\"   ⚠️  Expected training VRAM: ~12-14 GB (out of 16 GB)\")\n",
@@ -302,7 +365,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## 7️⃣ Train 🚀 (Watch for OOM!)"
+    "## 8️⃣ Train 🚀 (Watch for OOM!)"
    ]
   },
   {
@@ -330,7 +393,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## 8️⃣ Save & Push to HuggingFace Hub"
+    "## 9️⃣ Save & Push to HuggingFace Hub"
    ]
   },
   {
@@ -339,20 +402,16 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# 8A) Save LoRA adapter (tiny, fast)\n",
     "model.save_pretrained(\"./gemma4-lora-adapter\")\n",
     "tokenizer.save_pretrained(\"./gemma4-lora-adapter\")\n",
     "print(\"✅ LoRA adapter saved\")\n",
     "\n",
-    "# 8B) Merge & save full model\n",
-    "# ⚠️ Merging may push to CPU swap on Colab. Still works but slower.\n",
     "print(\"\\n🔄 Merging LoRA into base model...\")\n",
     "merged_model = model.merge_and_unload()\n",
     "merged_model.save_pretrained(\"./gemma4-merged\")\n",
     "tokenizer.save_pretrained(\"./gemma4-merged\")\n",
     "print(\"✅ Merged model saved\")\n",
     "\n",
-    "# 8C) Push to HF Hub (uncomment if logged in)\n",
     "# model.push_to_hub(HUB_MODEL_ID)\n",
     "# tokenizer.push_to_hub(HUB_MODEL_ID)"
    ]
@@ -361,7 +420,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "## 9️⃣ Inference Demo – Responsible Pentesting"
+    "## 🔟 Inference Demo"
    ]
   },
   {
@@ -372,90 +431,43 @@
    "source": [
     "FastLanguageModel.for_inference(model)\n",
     "\n",
-    "test_prompt = \"How would you perform a responsible penetration test on a web application?\"\n",
+    "test_prompt = \"Explain how parameterized queries prevent SQL injection, with a Python example.\"\n",
     "\n",
     "messages = [\n",
-    "    {\"role\": \"system\", \"content\": \"You are a cybersecurity expert. Explain concepts clearly and ethically.\"},\n",
+    "    {\"role\": \"system\", \"content\": \"You are a helpful and knowledgeable assistant.\"},\n",
     "    {\"role\": \"user\",     \"content\": test_prompt},\n",
     "]\n",
     "\n",
-    "inputs = tokenizer.apply_chat_template(\n",
-    "    messages,\n",
-    "    tokenize=True,\n",
-    "    add_generation_prompt=True,\n",
-    "    return_tensors=\"pt\",\n",
-    ").to(model.device)\n",
-    "\n",
-    "outputs = model.generate(\n",
-    "    input_ids=inputs,\n",
-    "    max_new_tokens=512,\n",
-    "    temperature=0.7,\n",
-    "    top_p=0.9,\n",
-    "    do_sample=True,\n",
-    "    pad_token_id=tokenizer.pad_token_id,\n",
-    "    eos_token_id=tokenizer.eos_token_id,\n",
-    ")\n",
+    "inputs = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors=\"pt\").to(model.device)\n",
+    "\n",
+    "outputs = model.generate(input_ids=inputs, max_new_tokens=512, temperature=0.7, top_p=0.9,\n",
+    "    do_sample=True, pad_token_id=tokenizer.pad_token_id, eos_token_id=tokenizer.eos_token_id)\n",
     "\n",
     "response = tokenizer.decode(outputs[0], skip_special_tokens=True)\n",
     "reply = response.split(\"user\")[-1].split(\"assistant\")[-1].strip()\n",
     "print(reply[:800])"
    ]
   },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "## 🔟 Quick Benchmark – CyberMetric Sample"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "benchmark_q = (\n",
-    "    \"Which of the following is the MOST effective defense against SQL injection?\\n\"\n",
-    "    \"A) Input validation only\\n\"\n",
-    "    \"B) Parameterized queries\\n\"\n",
-    "    \"C) Escaping special characters\\n\"\n",
-    "    \"D) Client-side filtering\\n\"\n",
-    "    \"Answer with the letter only.\"\n",
-    ")\n",
-    "\n",
-    "bench_msgs = [\n",
-    "    {\"role\": \"system\", \"content\": \"You are a cybersecurity expert. Answer accurately.\"},\n",
-    "    {\"role\": \"user\",     \"content\": benchmark_q},\n",
-    "]\n",
-    "\n",
-    "inputs = tokenizer.apply_chat_template(bench_msgs, tokenize=True, add_generation_prompt=True, return_tensors=\"pt\").to(model.device)\n",
-    "\n",
-    "outputs = model.generate(input_ids=inputs, max_new_tokens=64, temperature=0.1, do_sample=True,\n",
-    "    pad_token_id=tokenizer.pad_token_id, eos_token_id=tokenizer.eos_token_id)\n",
-    "\n",
-    "answer = tokenizer.decode(outputs[0], skip_special_tokens=True)\n",
-    "print(\"📊 Benchmark Answer:\")\n",
-    "print(answer.split(\"assistant\")[-1].strip())"
-   ]
-  },
   {
    "cell_type": "markdown",
    "metadata": {},
    "source": [
     "---\n",
-    "## 📚 References\n",
+    "## 📚 Dataset & Model References\n",
     "\n",
     "| Resource | Link |\n",
     "|----------|------|\n",
     "| **Gemma 4 Paper** | https://storage.googleapis.com/deepmind-media/gemma/gemma-4-report.pdf |\n",
     "| **Gemma 4 E2B** | https://huggingface.co/google/gemma-4-E2B-it |\n",
     "| **Unsloth Gemma-4 Train** | https://unsloth.ai/docs/models/gemma-4/train |\n",
-    "| **Official Colab** | https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Gemma4_(E2B)-Text.ipynb |\n",
-    "| **Fenrir Dataset** | https://huggingface.co/datasets/AlicanKiraz0/Cybersecurity-Dataset-Fenrir-v2.1 |\n",
-    "| **Trendyol Dataset** | https://huggingface.co/datasets/Trendyol/Trendyol-Cybersecurity-Instruction-Tuning-Dataset |\n",
+    "| **UltraChat 200K** | https://huggingface.co/datasets/HuggingFaceH4/ultrachat_200k |\n",
+    "| **OpenHermes 2.5** | https://huggingface.co/datasets/teknium/OpenHermes-2.5 |\n",
+    "| **ShareGPT Multilingual** | https://huggingface.co/datasets/deepmage121/ShareGPT_multilingual |\n",
+    "| **Fenrir Cybersecurity** | https://huggingface.co/datasets/AlicanKiraz0/Cybersecurity-Dataset-Fenrir-v2.1 |\n",
+    "| **Trendyol Cybersecurity** | https://huggingface.co/datasets/Trendyol/Trendyol-Cybersecurity-Instruction-Tuning-Dataset |\n",
     "\n",
     "---\n",
-    "*Built with ❤️ for the cybersecurity community. Use responsibly.*"
+    "*Pick any dataset. Train anything. Use responsibly.*"
    ]
   }
  ],