HIKARI-Vega-8B-SkinCaption-Fused-LoRA

---

🔌 Model Type: LoRA Adapter

This is a LoRA adapter (~1.1 GB) — it must be loaded on top of the merged Stage 2 model E27085921/HIKARI-Sirius-8B-SkinDx-RAG.

⚠️ Important: Do NOT load this on top of raw Qwen/Qwen3-VL-8B-Thinking. The merged-init design requires disease knowledge to already be in the base weights. Use HIKARI-Sirius-8B-SkinDx-RAG (merged) as the base.

✅ Advantage: Lightweight — download only ~1.1 GB instead of ~17 GB (plus the ~17 GB Stage 2 base).

💾 If you prefer a standalone ready-to-use model, see the merged version: E27085921/HIKARI-Vega-8B-SkinCaption-Fused (~17 GB)

---

What is this adapter?

LoRA adapter for HIKARI-Vega-8B-SkinCaption-Fused — Clinical skin lesion caption generation with merged-init strategy (best caption model). Metric: BLEU-4: 29.33 ⭐.

This is the LoRA adapter for the best caption generation model in the HIKARI family. Note: for correct behavior, load this adapter on top of the merged Stage 2 model (HIKARI-Sirius-8B-SkinDx-RAG), not the raw base model — disease knowledge must already be in the base weights.

See the full model card at E27085921/HIKARI-Vega-8B-SkinCaption-Fused for complete details, usage examples, and performance comparison.

---

Usage

from peft import PeftModel
from transformers import Qwen3VLForConditionalGeneration, AutoProcessor
import torch
from PIL import Image

# Step 1: Load the MERGED Stage 2 disease model as base
#         (disease knowledge is permanently in its weights — DO NOT use raw Qwen base here)
base = Qwen3VLForConditionalGeneration.from_pretrained(
    "E27085921/HIKARI-Sirius-8B-SkinDx-RAG",   # merged Stage 2 weights (~17 GB)
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

# Step 2: Apply Stage 3 caption LoRA adapter (~1.1 GB)
model = PeftModel.from_pretrained(base, "E27085921/HIKARI-Vega-8B-SkinCaption-Fused-LoRA")
processor = AutoProcessor.from_pretrained("E27085921/HIKARI-Vega-8B-SkinCaption-Fused-LoRA", trust_remote_code=True)

# Step 3: Inference — see full examples at E27085921/HIKARI-Vega-8B-SkinCaption-Fused
image = Image.open("skin_lesion.jpg").convert("RGB")

For complete inference examples including vLLM and SGLang production code, see: E27085921/HIKARI-Vega-8B-SkinCaption-Fused

---

📄 Citation

@misc{hikari2026,
  title  = {HIKARI: RAG-in-Training for Skin Disease Diagnosis
            with Cascaded Vision-Language Models},
  author = {Watin Promfiy and Pawitra Boonprasart},
  year   = {2026},
  institution = {King Mongkut's Institute of Technology Ladkrabang,
                 Department of Information Technology, Bangkok, Thailand}
}

Made with ❤️ at King Mongkut's Institute of Technology Ladkrabang (KMITL)