license: apache-2.0
library_name: peft
base_model: lmsys/vicuna-7b-v1.5
pipeline_tag: image-to-text
tags:
- medical-imaging
- chest-xray
- mimic-cxr
- vision-language
- fairness
- lora
- peft
datasets:
- physionet/mimic-cxr-jpg
FairLLaVA — MIMIC-CXR
Fairness-aware LoRA adapter on top of LLaVA-Rad (Vicuna-7B + BiomedCLIP-CXR-518) for MIMIC-CXR chest-X-ray report generation. Trained with the FairLLaVA mutual-information regularizer on patient demographics (age, sex, race) to reduce inter-group performance gaps while preserving clinical accuracy.
Code: github.com/bhosalems/FairLLaVA Paper: arxiv.org/abs/2603.26008
Files in this directory
| File | Purpose |
|---|---|
adapter_model.safetensors, adapter_config.json |
LoRA adapter weights + config |
non_lora_trainables.bin |
non-LoRA trainable params (projector + token embeddings) |
mm_projector.bin |
multimodal projector (vision -> LLM token space) |
config.json |
LLaVA model config |
tokenizer.model, tokenizer_config.json, special_tokens_map.json |
Vicuna tokenizer |
Quick start
from huggingface_hub import snapshot_download
from llava.model.builder import load_pretrained_model
local_dir = snapshot_download(
repo_id="mbhosale/FairLLaVA",
allow_patterns="mimic-cxr/*",
)
tokenizer, model, image_processor, ctx_len = load_pretrained_model(
f"{local_dir}/mimic-cxr",
model_base="lmsys/vicuna-7b-v1.5",
model_name="llavarad",
)
See the full inference example in inference.py.
Ethics
This checkpoint is released for research and educational use only. It is not approved or validated for clinical or diagnostic use and must not be used to make medical decisions or to inform patient care. Use of MIMIC-CXR is governed by the PhysioNet data-use agreement.
Citation
If you use this checkpoint, please cite FairLLaVA and the upstream works it builds on:
@article{bhosale2026fairllava,
title={FairLLaVA: Fairness-Aware Parameter-Efficient Fine-Tuning for Large Vision-Language Assistants},
author={Bhosale, Mahesh and Wasi, Abdul and Srivastava, Shantam and Latif, Shifa and Luan, Tianyu and Gao, Mingchen and Doermann, David and Gong, Xuan},
journal={arXiv preprint arXiv:2603.26008},
year={2026}
}