mimic-cxr/README.md

---
license: apache-2.0
library_name: peft
base_model: lmsys/vicuna-7b-v1.5
pipeline_tag: image-to-text
tags:
  - medical-imaging
  - chest-xray
  - mimic-cxr
  - vision-language
  - fairness
  - lora
  - peft
datasets:
  - physionet/mimic-cxr-jpg
---

# FairLLaVA — MIMIC-CXR

Fairness-aware LoRA adapter on top of LLaVA-Rad (Vicuna-7B + BiomedCLIP-CXR-518)
for **MIMIC-CXR** chest-X-ray report generation. Trained with the FairLLaVA
mutual-information regularizer on patient demographics (age, sex, race) to
reduce inter-group performance gaps while preserving clinical accuracy.

Code: [github.com/bhosalems/FairLLaVA](https://github.com/bhosalems/FairLLaVA)
Paper: [arxiv.org/abs/2603.26008](https://arxiv.org/abs/2603.26008)

## Files in this directory

| File | Purpose |
|---|---|
| `adapter_model.safetensors`, `adapter_config.json` | LoRA adapter weights + config |
| `non_lora_trainables.bin` | non-LoRA trainable params (projector + token embeddings) |
| `mm_projector.bin` | multimodal projector (vision -> LLM token space) |
| `config.json` | LLaVA model config |
| `tokenizer.model`, `tokenizer_config.json`, `special_tokens_map.json` | Vicuna tokenizer |

## Quick start

```python
from huggingface_hub import snapshot_download
from llava.model.builder import load_pretrained_model

local_dir = snapshot_download(
    repo_id="mbhosale/FairLLaVA",
    allow_patterns="mimic-cxr/*",
)

tokenizer, model, image_processor, ctx_len = load_pretrained_model(
    f"{local_dir}/mimic-cxr",
    model_base="lmsys/vicuna-7b-v1.5",
    model_name="llavarad",
)
```

See the full inference example in [`inference.py`](https://github.com/bhosalems/FairLLaVA/blob/main/inference.py).

## Ethics

This checkpoint is released **for research and educational use only**. It is
**not** approved or validated for clinical or diagnostic use and must not be
used to make medical decisions or to inform patient care. Use of MIMIC-CXR is
governed by the PhysioNet data-use agreement.

## Citation

If you use this checkpoint, please cite FairLLaVA and the upstream works it builds on:

```bibtex
@article{bhosale2026fairllava,
  title={FairLLaVA: Fairness-Aware Parameter-Efficient Fine-Tuning for Large Vision-Language Assistants},
  author={Bhosale, Mahesh and Wasi, Abdul and Srivastava, Shantam and Latif, Shifa and Luan, Tianyu and Gao, Mingchen and Doermann, David and Gong, Xuan},
  journal={arXiv preprint arXiv:2603.26008},
  year={2026}
}
```