mou11/medical-mistral-7b-qlora

🏥 Medical Mistral-7B QLoRA

A QLoRA fine-tuned version of Mistral-7B-Instruct-v0.2 on medical question answering data (USMLE-style clinical vignettes).

A QLoRA fine-tuned version of Mistral-7B-Instruct-v0.2 on medical question answering data (USMLE-style clinical vignettes).

Model Description

• Base Model: mistralai/Mistral-7B-Instruct-v0.2
• Fine-tuning Method: QLoRA (4-bit NF4 quantization + LoRA adapters)
• Dataset: medalpaca/medical_meadow_medqa (9,160 training samples)
• Task: Medical multiple choice question answering
• Trainable Parameters: 13.6M (0.19% of 7.2B total)
• Training Platform: Kaggle T4 GPU (free tier)
• Experiment Tracking: Weights & Biases

Training Details

Hyperparameter	Value
LoRA Rank (r)	16
LoRA Alpha	32
Target Modules	q_proj, k_proj, v_proj, o_proj
Learning Rate	2e-4
Batch Size	2
Gradient Accumulation	8 steps
Epochs	1
Optimizer	paged_adamw_8bit

Results

Metric	Value
Training Loss	1.039
Validation Loss	0.987
USMLE Sample Accuracy	30% (beats 20% random baseline)

How to Use

from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel
import torch

base_model = "mistralai/Mistral-7B-Instruct-v0.2"
adapter = "mou11/medical-mistral-7b-qlora"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.float16,
)

model = AutoModelForCausalLM.from_pretrained(
    base_model,
    quantization_config=bnb_config,
    device_map="auto"
)
model = PeftModel.from_pretrained(model, adapter)
tokenizer = AutoTokenizer.from_pretrained(base_model)

prompt = """### Instruction:
Please answer with one of the option in the bracket

### Input:
Q: Your medical question here with options A/B/C/D/E

### Response:"""

inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=50, temperature=0.1)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Part of Medical AI Portfolio

This model is Project 3 of a 5-project Medical AI Engineering portfolio:

1. ✅ Pneumonia Detection (CNN + Grad-CAM)
2. ✅ Clinical Decision Support Agent (LangGraph + RAG)
3. ✅ Medical LLM Fine-tuning (QLoRA) — this model
4. 🔄 Medical Report Generator (Hallucination Detection + FHIR)
5. 🔄 Model Optimization Pipeline (ONNX + TensorRT)

Limitations

• Trained for 1 epoch due to compute constraints
• Not suitable for real clinical use without further validation
• Performance improves significantly with more training epochs