Clarke LoRA Adapter - MedGemma 27B for NHS Clinic Letters
QLoRA fine-tuned adapter for google/medgemma-27b-text-it.
Built for the MedGemma Impact Challenge on Kaggle. Part of the Clarke clinical documentation system.
Training Details
- Method: QLoRA (4-bit NF4, LoRA rank 16, alpha 32, dropout 0.05)
- Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj (attention + MLP)
- Training data: 5 gold-standard NHS clinic letters (separate Assessment, Plan, and Advice to Patient sections)
- Epochs: 3
- Learning rate: 2e-4, 8-bit AdamW optimiser
- Hardware: NVIDIA A100 80 GB (HuggingFace Spaces)
- Training loss: 2.09 to 1.30 (38% reduction)
- Competition: MedGemma Impact Challenge
Evaluation
The adapter was evaluated against FHIR-aligned gold-standard NHS clinic letters across five patients. The base model with optimised prompting outperformed the adapter at this data scale (n=5):
| Metric | Base Model | LoRA Adapter |
|---|---|---|
| BLEU-1 | 0.82 | 0.79 |
| BLEU-4 | 0.61 | 0.39 |
| ROUGE-L | 0.74 | 0.60 |
This result is consistent with the literature on fine-tuning large language models with very small datasets. The adapter demonstrates the fine-tuning pipeline and would benefit from a larger clinical corpus (50-200+ examples). Full evaluation methodology is in the Clarke evaluation report.
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_model = AutoModelForCausalLM.from_pretrained("google/medgemma-27b-text-it")
model = PeftModel.from_pretrained(base_model, "yashvshetty/clarke-medgemma-27b-lora")
Links
- Downloads last month
- 7
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support