Update combined model card

README.md

@@ -4,7 +4,9 @@ language:
   - en
 library_name: peft
 pipeline_tag: text-generation
-base_model: Qwen/Qwen2.5-1.5B-Instruct
+base_model:
+  - Qwen/Qwen2.5-1.5B-Instruct
+  - meta-llama/Llama-3.2-1B-Instruct
 tags:
   - medical
   - healthcare
@@ -14,26 +16,43 @@ tags:
   - lora
   - qwen
   - qwen2.5
+  - llama
+  - llama-3.2
   - ollama
   - gguf
 ---
 
-# Qwen2.5-1.5B Medical QA (QLoRA)
+# Healthcare LLM Assistant — QLoRA fine-tunes
 
-QLoRA fine-tune of [`Qwen/Qwen2.5-1.5B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct) on a 9,000-pair mix of six public biomedical Q&A sources. Built as part of the COMP8420 (Macquarie University) main project on a healthcare NLP assistant. The fine-tuned model is served locally via Ollama and benchmarked head-to-head against GPT-5.5 in the parent GitHub repo.
+Two parallel QLoRA fine-tunes of small instruct models on the same 9,000-pair mix of public biomedical Q&A, served side-by-side in the parent project's Streamlit UI for a 3-way bake-off against GPT-5.5.
 
-**Companion code**: https://github.com/NhatNguyen3001/COMP8420-Healthcare-LLM-Assistant
+| Variant | Subfolder | Base | Adapter | GGUF (Q4_K_M) |
+|---|---|---|---|---|
+| **Qwen** | `qwen/` | [`Qwen/Qwen2.5-1.5B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct) | `qwen/qwen-medqa-adapter/` (~82 MB) | `qwen/qwen-medqa-gguf/model.Q4_K_M.gguf` (~941 MB) |
+| **Llama-3.2** | `llama32/` | [`meta-llama/Llama-3.2-1B-Instruct`](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct) | `llama32/llama32-medqa-adapter/` (~50 MB) | `llama32/llama32-medqa-gguf/model.Q4_K_M.gguf` (~770 MB) |
+
+Both variants were trained with the same dataset, the same LoRA shape (r=16, α=32, all 7 projection layers) and the same SFT recipe, so any quality gap isolates the base-model effect.
+
+Built as part of the COMP8420 (Macquarie University) main project on a healthcare NLP assistant. Companion code: **https://github.com/NhatNguyen3001/COMP8420-Healthcare-LLM-Assistant**
 (see the GitHub README for the full system: voice input, PII railguard, multi-agent RAG, evaluation notebooks.)
 
 ## What is in this repo
 
-| Path | Size | What |
-|---|---|---|
-| `qwen-medqa-adapter/` | ~82 MB | PEFT LoRA adapter (re-apply to base Qwen2.5-1.5B-Instruct with `peft`) |
-| `qwen-medqa-gguf/model.Q4_K_M.gguf` | ~941 MB | Merged + Q4_K_M quantized model, ready for Ollama or llama.cpp |
-| `qwen-medqa-gguf/Modelfile` | <1 KB | Ollama registration recipe |
+```
+.
+├── qwen/
+│   ├── qwen-medqa-adapter/                  # PEFT LoRA adapter
+│   └── qwen-medqa-gguf/
+│       ├── model.Q4_K_M.gguf                # Ollama-ready GGUF
+│       └── Modelfile                        # Ollama registration recipe
+└── llama32/
+    ├── llama32-medqa-adapter/               # PEFT LoRA adapter
+    └── llama32-medqa-gguf/
+        ├── model.Q4_K_M.gguf
+        └── Modelfile
+```
 
-The merged-but-unquantized `safetensors` is intentionally not uploaded; it is redundant for end users (use the GGUF for Ollama OR the adapter for transformers+peft).
+The merged-but-unquantized `safetensors` is intentionally not uploaded for either variant; it is redundant for end users (use the GGUF for Ollama OR the adapter for transformers+peft).
 
 ## Training data
 
@@ -48,13 +67,14 @@ The merged-but-unquantized `safetensors` is intentionally not uploaded; it is re
 | DrugBank `side_effects` | ~1,500 | side-effect summaries |
 | DrugBank `mechanism_of_action` | ~1,500 | MoA explanations |
 
-90 / 5 / 5 random split with `seed=42`. The OpenAI messages format was used at JSONL level; the Qwen2.5 chat template is applied at training time, not stored in the JSONL.
+90 / 5 / 5 random split with `seed=42`. The OpenAI messages format is used at JSONL level; each variant's chat template (Qwen2.5 or Llama-3.1) is applied at training time, not stored in the JSONL.
 
 ## Training setup
 
+Same hyperparameters across both variants:
+
 | Hyperparameter | Value |
 |---|---|
-| Base | `Qwen/Qwen2.5-1.5B-Instruct` (4-bit NF4 via bitsandbytes) |
 | LoRA rank `r` | 16 |
 | LoRA alpha | 32 |
 | LoRA target modules | all 7 projection layers (q, k, v, o, gate, up, down) |
@@ -66,9 +86,17 @@ The merged-but-unquantized `safetensors` is intentionally not uploaded; it is re
 | Optimizer | `adamw_8bit` |
 | Seed | 42 |
 | Hardware | RTX 4060 (8 GB, bf16) |
-| Wall time | ~5,667 seconds (~95 minutes) |
 
-Best validation loss: 1.5536 around epoch 1.98. The deployed checkpoint is end-of-epoch-3 (the "what a full QLoRA run gives you" baseline, not early-stopped).
+Per-variant differences:
+
+| | Qwen | Llama-3.2 |
+|---|---|---|
+| Base id | `Qwen/Qwen2.5-1.5B-Instruct` (4-bit NF4) | `meta-llama/Llama-3.2-1B-Instruct` (4-bit NF4) |
+| Chat template | `qwen-2.5` | `llama-3.1` |
+| Wall time (3 epochs) | ~95 min | ~30-45 min (smaller base) |
+| Best val loss | 1.5536 (~epoch 1.98) | see `results/training_qlora_llama32.md` in parent repo |
+
+Deployed checkpoints are end-of-epoch-3 for both (the "what a full QLoRA run gives you" baseline, not early-stopped).
 
 ## Evaluation
 
@@ -79,13 +107,15 @@ Two evaluation passes:
 1. **Surface metrics**: ROUGE-1/2/L + BERTScore-F1 (with the PubMedBERT backbone)
 2. **LLM-as-judge**: GPT-5.4 scoring blind on Accuracy / Completeness / Clarity / Safety (0-10), reference-aware
 
-**Headline findings (vs GPT-5.5):**
+**Headline findings (vs GPT-5.5), Qwen variant:**
 
-- This QLoRA model wins ROUGE-L by ~+0.022 (~+12% relative) and BERTScore-F1 by ~+0.0067 (~+0.8% relative)
+- The Qwen QLoRA model wins ROUGE-L by ~+0.022 (~+12% relative) and BERTScore-F1 by ~+0.0067 (~+0.8% relative) against GPT-5.5
 - The win is driven by **template substitution**, not factual improvement. The training set includes 71+ DrugBank entries sharing the skeleton "`{X}` pollen is the pollen of the `{X}` plant. `{X}` pollen is mainly used in allergenic testing." The fine-tune learns the template and slot-fills the entity at inference; ROUGE and BERTScore both reward this even when the substituted entity is wrong.
 - Verified 0 / 450 literal Q+A pair overlap between train and test, so this is template generalization, not memorization.
 - Under the LLM-as-judge Accuracy dimension, GPT-5.5 leads (judge results in the parent repo's `results/llm_judge_evaluation.csv`).
 
+**Llama-3.2 variant:** see the 3-way numbers in the parent repo's `results/model_comparison.csv` (refreshed after the Llama run). The same template-substitution dynamic is expected on shared DrugBank slots; the contrast with the Qwen variant isolates the base-model contribution.
+
 Detailed numbers and charts live in the parent repo:
 
 - `results/llm_generation_evaluation.csv` + `llm_generation_eval_chart.png` + `llm_generation_bertscore_chart.png`
@@ -95,34 +125,45 @@ Detailed numbers and charts live in the parent repo:
 
 ## How to use
 
+Replace `<variant>` with `qwen` or `llama32` in the examples below.
+
 ### Option 1 — Ollama (recommended for local serving)
 
 ```bash
-# Fetch the GGUF + Modelfile
+# Fetch one variant's GGUF + Modelfile
 huggingface-cli download Davis426/COMP8420-Healthcare-LLM-Assistant \
-  --include "qwen-medqa-gguf/*" \
+  --include "qwen/qwen-medqa-gguf/*" \
   --local-dir ./models
 
 # Register with Ollama
-cd ./models/qwen-medqa-gguf
+cd ./models/qwen/qwen-medqa-gguf
 ollama create medqa-qwen -f Modelfile
 
 # Try it
 ollama run medqa-qwen "What is amoxicillin used for?"
 ```
 
+For the Llama variant, swap every `qwen` for `llama32` (paths) and the Ollama tag to `medqa-llama32`.
+
+You can register both side-by-side; one `ollama serve` daemon handles both tags concurrently (`OLLAMA_MAX_LOADED_MODELS` defaults to 3).
+
 ### Option 2 — transformers + peft (Python)
 
 ```python
 from peft import PeftModel
 from transformers import AutoModelForCausalLM, AutoTokenizer
 
-base_id = "Qwen/Qwen2.5-1.5B-Instruct"
+# pick a variant
+base_id    = "Qwen/Qwen2.5-1.5B-Instruct"
+subfolder  = "qwen/qwen-medqa-adapter"
+# or:
+# base_id   = "meta-llama/Llama-3.2-1B-Instruct"
+# subfolder = "llama32/llama32-medqa-adapter"
 adapter_id = "Davis426/COMP8420-Healthcare-LLM-Assistant"
 
 tokenizer = AutoTokenizer.from_pretrained(base_id)
 base = AutoModelForCausalLM.from_pretrained(base_id, device_map="auto")
-model = PeftModel.from_pretrained(base, adapter_id, subfolder="qwen-medqa-adapter")
+model = PeftModel.from_pretrained(base, adapter_id, subfolder=subfolder)
 
 messages = [{"role": "user", "content": "What is amoxicillin used for?"}]
 inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
@@ -134,31 +175,33 @@ print(tokenizer.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))
 
 ```bash
 huggingface-cli download Davis426/COMP8420-Healthcare-LLM-Assistant \
-  --include "qwen-medqa-gguf/model.Q4_K_M.gguf" --local-dir .
+  --include "qwen/qwen-medqa-gguf/model.Q4_K_M.gguf" --local-dir .
 
-./llama-cli -m model.Q4_K_M.gguf -p "What is amoxicillin used for?" -n 256
+./llama-cli -m qwen/qwen-medqa-gguf/model.Q4_K_M.gguf \
+  -p "What is amoxicillin used for?" -n 256
 ```
 
 ## Limitations
 
-This model is a teaching / research artifact. **Do not use for real clinical decisions.** Specifically:
+Both models are teaching / research artifacts. **Do not use for real clinical decisions.** Specifically:
 
-- **Catastrophic forgetting on out-of-distribution prompts.** Fine-tuning on a narrow Q&A distribution at 1.5B parameter scale shifts the base model hard. Casual / non-medical questions get answered in MedQA-style; the base model's general conversational ability is degraded.
-- **Weakened in-context grounding.** Every training pair has shape `user_question -> answer`, with no retrieved-context block. As a result the fine-tuned model partly loses the ability to read RAG passages in the prompt and tends to answer from parametric memory even when correct evidence is supplied. The parent repo's MASS-RAG pipeline retains GPT-5.5 for cases where grounded answers matter; this local model is sidebar-selectable for the comparison experience.
-- **No factual safety net.** Both training data and evaluation rely on existing biomedical corpora; the model has no live knowledge cutoff or up-to-date drug-interaction database. The parent repo applies a regex-based PII railguard on user input, but the model output itself is not safety-filtered beyond what the base model already does.
+- **Catastrophic forgetting on out-of-distribution prompts.** Fine-tuning on a narrow Q&A distribution at the 1-1.5B parameter scale shifts each base model hard. Casual / non-medical questions get answered in MedQA-style; the base model's general conversational ability is degraded.
+- **Weakened in-context grounding.** Every training pair has shape `user_question -> answer`, with no retrieved-context block. As a result both fine-tuned models partly lose the ability to read RAG passages in the prompt and tend to answer from parametric memory even when correct evidence is supplied. The parent repo's MASS-RAG pipeline retains GPT-5.5 for cases where grounded answers matter; the local models are sidebar-selectable for the comparison experience.
+- **No factual safety net.** Both training data and evaluation rely on existing biomedical corpora; the models have no live knowledge cutoff or up-to-date drug-interaction database. The parent repo applies a regex-based PII railguard on user input, but model output itself is not safety-filtered beyond what each base model already does.
 - **English only.**
+- **Llama-3.2 base licence:** Llama-3.2 community licence applies to the Llama variant (acceptance via the gated HF repo); see the Meta licence for permitted uses.
 
 ## License
 
-`cc-by-nc-4.0` — research and non-commercial use. The base model (Qwen2.5-1.5B-Instruct) is Apache-2.0. Downstream dataset licenses may impose additional restrictions; please consult each source (BioASQ, MedQuAD, DrugBank, MedRAG textbooks) before redistribution.
+The fine-tuned adapters and GGUFs in this repo are released under `cc-by-nc-4.0` (research and non-commercial use). Base model licences override where stricter: Qwen2.5 is Apache-2.0; Llama-3.2 is under the Meta Llama 3.2 Community Licence. Downstream dataset licences may impose additional restrictions; please consult each source (BioASQ, MedQuAD, DrugBank, MedRAG textbooks) before redistribution.
 
 ## Citation
 
 If you use or build on this work, please reference:
 
 ```bibtex
-@misc{comp8420-2026-medqa-qwen,
-  title  = {Healthcare NLP Assistant: QLoRA-fine-tuned Qwen2.5-1.5B for medical Q&A},
+@misc{comp8420-2026-medqa,
+  title  = {Healthcare NLP Assistant: parallel QLoRA fine-tunes of Qwen2.5-1.5B and Llama-3.2-1B for medical Q&A},
   author = {Davis426},
   year   = {2026},
   howpublished = {\url{https://huggingface.co/Davis426/COMP8420-Healthcare-LLM-Assistant}}
@@ -168,7 +211,8 @@ If you use or build on this work, please reference:
 Built on top of:
 
 - Qwen2.5 (Alibaba): https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct
+- Llama-3.2 (Meta): https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct
 - QLoRA (Dettmers et al., 2023): https://arxiv.org/abs/2305.14314
-- MASS-RAG (Xiao, Huang, Liu, Xie, 2026): https://arxiv.org/abs/2604.18509 (used by the parent repo's retrieval pipeline that this model plugs into)
+- MASS-RAG (Xiao, Huang, Liu, Xie, 2026): https://arxiv.org/abs/2604.18509 (used by the parent repo's retrieval pipeline that these models plug into)
 - Unsloth: https://github.com/unslothai/unsloth
 - llama.cpp + Ollama for GGUF serving