mbhosale commited on
Commit
77a28d9
·
verified ·
1 Parent(s): 25fce7f

Add MIMIC-CXR subdir README with citations

Browse files
Files changed (1) hide show
  1. mimic-cxr/README.md +89 -9
mimic-cxr/README.md CHANGED
@@ -2,25 +2,105 @@
2
  license: apache-2.0
3
  library_name: peft
4
  base_model: lmsys/vicuna-7b-v1.5
 
5
  tags:
6
  - medical-imaging
7
  - chest-xray
8
  - mimic-cxr
 
9
  - fairness
10
  - lora
 
 
 
11
  ---
12
 
13
  # FairLLaVA — MIMIC-CXR
14
 
15
  Fairness-aware LoRA adapter on top of LLaVA-Rad (Vicuna-7B + BiomedCLIP-CXR-518)
16
- for MIMIC-CXR chest-X-ray report generation. Trained with the FairLLaVA
17
- mutual-information regularizer on patient demographics (age, sex, race).
 
18
 
19
- ## Files
20
- - `adapter_model.safetensors`, `adapter_config.json`, `non_lora_trainables.bin` — LoRA weights
21
- - `mm_projector.bin` — multimodal projector
22
- - `tokenizer.model`, `tokenizer_config.json`, `special_tokens_map.json`, `config.json` — tokenizer / config
23
 
24
- ## Usage
25
- See https://github.com/bhosalems/FairLLaVA → `inference.py`. Point `model_path` at
26
- this directory and set `model_base="lmsys/vicuna-7b-v1.5"`.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
  license: apache-2.0
3
  library_name: peft
4
  base_model: lmsys/vicuna-7b-v1.5
5
+ pipeline_tag: image-to-text
6
  tags:
7
  - medical-imaging
8
  - chest-xray
9
  - mimic-cxr
10
+ - vision-language
11
  - fairness
12
  - lora
13
+ - peft
14
+ datasets:
15
+ - physionet/mimic-cxr-jpg
16
  ---
17
 
18
  # FairLLaVA — MIMIC-CXR
19
 
20
  Fairness-aware LoRA adapter on top of LLaVA-Rad (Vicuna-7B + BiomedCLIP-CXR-518)
21
+ for **MIMIC-CXR** chest-X-ray report generation. Trained with the FairLLaVA
22
+ mutual-information regularizer on patient demographics (age, sex, race) to
23
+ reduce inter-group performance gaps while preserving clinical accuracy.
24
 
25
+ Code: [github.com/bhosalems/FairLLaVA](https://github.com/bhosalems/FairLLaVA)
26
+ Paper: [arxiv.org/abs/2603.26008](https://arxiv.org/abs/2603.26008)
 
 
27
 
28
+ ## Files in this directory
29
+
30
+ | File | Purpose |
31
+ |---|---|
32
+ | `adapter_model.safetensors`, `adapter_config.json` | LoRA adapter weights + config |
33
+ | `non_lora_trainables.bin` | non-LoRA trainable params (projector + token embeddings) |
34
+ | `mm_projector.bin` | multimodal projector (vision -> LLM token space) |
35
+ | `config.json` | LLaVA model config |
36
+ | `tokenizer.model`, `tokenizer_config.json`, `special_tokens_map.json` | Vicuna tokenizer |
37
+
38
+ ## Quick start
39
+
40
+ ```python
41
+ from huggingface_hub import snapshot_download
42
+ from llava.model.builder import load_pretrained_model
43
+
44
+ local_dir = snapshot_download(
45
+ repo_id="mbhosale/FairLLaVA",
46
+ allow_patterns="mimic-cxr/*",
47
+ )
48
+
49
+ tokenizer, model, image_processor, ctx_len = load_pretrained_model(
50
+ f"{local_dir}/mimic-cxr",
51
+ model_base="lmsys/vicuna-7b-v1.5",
52
+ model_name="llavarad",
53
+ )
54
+ ```
55
+
56
+ See the full inference example in [`inference.py`](https://github.com/bhosalems/FairLLaVA/blob/main/inference.py).
57
+
58
+ ## Ethics
59
+
60
+ This checkpoint is released **for research and educational use only**. It is
61
+ **not** approved or validated for clinical or diagnostic use and must not be
62
+ used to make medical decisions or to inform patient care. Use of MIMIC-CXR is
63
+ governed by the PhysioNet data-use agreement.
64
+
65
+ ## Citation
66
+
67
+ If you use this checkpoint, please cite FairLLaVA and the upstream works it builds on:
68
+
69
+ ```bibtex
70
+ @misc{bhosale2026fairllava,
71
+ title={FairLLaVA: Fairness-Aware Parameter-Efficient Fine-Tuning for Large Vision-Language Assistants},
72
+ author={Mahesh Bhosale and Abdul Wasi and Shantam Srivastava and Shifa Latif and Tianyu Luan and Mingchen Gao and David Doermann and Xuan Gong},
73
+ year={2026},
74
+ eprint={2603.26008},
75
+ archivePrefix={arXiv},
76
+ primaryClass={cs.CV},
77
+ url={https://arxiv.org/abs/2603.26008}
78
+ }
79
+
80
+ @article{ZambranoChaves2025,
81
+ title={A clinically accessible small multimodal radiology model and evaluation metric for chest X-ray findings},
82
+ author={Zambrano Chaves, Juan Manuel and others},
83
+ journal={Nature Communications}, year={2025}, volume={16}, pages={3108},
84
+ doi={10.1038/s41467-025-58344-x}
85
+ }
86
+
87
+ @misc{liu2023improvedllava,
88
+ title={Improved Baselines with Visual Instruction Tuning},
89
+ author={Liu, Haotian and Li, Chunyuan and Li, Yuheng and Lee, Yong Jae},
90
+ publisher={arXiv:2310.03744},
91
+ year={2023}
92
+ }
93
+
94
+ @article{johnson2019mimic,
95
+ title={MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports},
96
+ author={Johnson, Alistair E. W. and Pollard, Tom J. and Berkowitz, Seth J. and Greenbaum, Nathaniel R. and Lungren, Matthew P. and Deng, Chih-ying and Mark, Roger G. and Horng, Steven},
97
+ journal={Scientific Data}, year={2019}, volume={6}, number={1}, pages={317},
98
+ doi={10.1038/s41597-019-0322-0}
99
+ }
100
+
101
+ @article{johnson2019mimiccxrjpg,
102
+ title={MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs},
103
+ author={Johnson, Alistair E. W. and Pollard, Tom J. and Greenbaum, Nathaniel R. and Lungren, Matthew P. and Deng, Chih-ying and Peng, Yifan and Lu, Zhiyong and Mark, Roger G. and Berkowitz, Seth J. and Horng, Steven},
104
+ journal={arXiv preprint arXiv:1901.07042}, year={2019}
105
+ }
106
+ ```