giuseppericcio commited on
Commit
c70da53
Β·
verified Β·
1 Parent(s): 54f3761

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +133 -0
README.md CHANGED
@@ -1,3 +1,136 @@
1
  ---
2
  license: cc-by-nc-4.0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: cc-by-nc-4.0
3
+ language:
4
+ - it
5
+ tags:
6
+ - llama
7
+ - llama-3
8
+ - meta
9
+ - medical-qa
10
+ - italian
11
+ - biomedical
12
+ - question-answering
13
+ - fine-tuning
14
+ - unsloth
15
+ - bnb
16
+ - 4bit
17
+ - imb
18
+ - Cardiologia
19
+ datasets:
20
+ - praiselab-picuslab/IMB
21
+ base_model:
22
+ - unsloth/Llama-3.2-1B-Instruct-unsloth-bnb-4bit
23
  ---
24
+
25
+ # 🧠 Llama-3.2-1B-Instruct β€” IMB Cardiologia Fine-Tuned Model
26
+
27
+ This model is a fine-tuned version of [`unsloth/Llama-3.2-1B-Instruct-unsloth-bnb-4bit`](https://huggingface.co/unsloth/Llama-3.2-1B-Instruct-unsloth-bnb-4bit), optimized for **Italian medical question answering**, with a specific focus on **Cardiologia**.
28
+
29
+ The fine-tuning was performed using a **subset of the IMB (Italian Medical Benchmark) dataset**, specifically:
30
+ - Cardiologia category only
31
+ - ~10,000 training samples
32
+
33
+ The training was performed using the **Unsloth** library with LoRA fine-tuning, and the adapter weights were later merged into the base model to provide a standalone checkpoint.
34
+
35
+ This model relies on data from the IMB dataset. **If you use this model in research or applications, you must cite the IMB paper (see Citation section below).**
36
+
37
+ ---
38
+
39
+ ## πŸ“š Training Dataset β€” IMB (Italian Medical Benchmark)
40
+
41
+ IMB is an Italian benchmark for medical question answering, designed to evaluate and improve LLM performance in clinical-domain Italian language understanding and reasoning.
42
+
43
+ The full dataset includes:
44
+
45
+ - **IMB-QA**: 782,644 doctor-patient conversations collected from Italian online medical forums
46
+ - **IMB-MCQA**: 25,862 multiple-choice questions derived from Italian medical specialization exams
47
+
48
+ ⚠️ **Important:**
49
+ This model was trained **only on the Cardiologia subset (~10,000 samples)** of IMB, not on the full dataset.
50
+
51
+ Dataset repository:
52
+ πŸ‘‰ https://github.com/PRAISELab-PicusLab/IMB
53
+
54
+ ---
55
+
56
+ ## πŸ§ͺ Usage Example
57
+
58
+ ```python
59
+ from transformers import AutoModelForCausalLM, AutoTokenizer
60
+
61
+ model = AutoModelForCausalLM.from_pretrained("praiselab-picuslab/Llama-3.2-1B-Instruct-Cardiologia")
62
+ tokenizer = AutoTokenizer.from_pretrained("praiselab-picuslab/Llama-3.2-1B-Instruct-Cardiologia")
63
+
64
+ prompt = "[Example question in Italian about Cardiologia]"
65
+ inputs = tokenizer(prompt, return_tensors="pt")
66
+ outputs = model.generate(**inputs, max_new_tokens=150)
67
+
68
+ print(tokenizer.decode(outputs[0], skip_special_tokens=True))
69
+ ```
70
+
71
+ ---
72
+
73
+ ## ⚠️ Usage Restrictions
74
+
75
+ * Allowed use: **Non-commercial research only**
76
+ * Redistribution: Not allowed without explicit authorization
77
+ * Mandatory citation: The IMB dataset paper must be cited in any publication or derived work
78
+
79
+ ---
80
+
81
+ ## πŸ“„ Citation
82
+
83
+ If you use this model, the IMB dataset, or derived outputs in research, please cite:
84
+
85
+ ```bibtex
86
+ @inproceedings{DBLP:conf/clic-it/RomanoRBPM25,
87
+ author = {Antonio Romano and
88
+ Giuseppe Riccio and
89
+ Mariano Barone and
90
+ Marco Postiglione and
91
+ Vincenzo Moscato},
92
+ editor = {Cristina Bosco and
93
+ Elisabetta Jezek and
94
+ Marco Polignano and
95
+ Manuela Sanguinetti},
96
+ title = {{IMB:} An Italian Medical Benchmark for Question Answering},
97
+ booktitle = {Proceedings of the Eleventh Italian Conference on Computational Linguistics
98
+ (CLiC-it 2025), Cagliari, Italy, September 24-26, 2025},
99
+ series = {{CEUR} Workshop Proceedings},
100
+ volume = {4112},
101
+ publisher = {CEUR-WS.org},
102
+ year = {2025},
103
+ url = {https://ceur-ws.org/Vol-4112/92_main_long.pdf}
104
+ }
105
+ ```
106
+
107
+ ---
108
+
109
+ ## πŸ— Training Details
110
+
111
+ * Base model: `unsloth/Llama-3.2-1B-Instruct-unsloth-bnb-4bit`
112
+ * Fine-tuning method: LoRA (Unsloth)
113
+ * Quantization: 4-bit (BitsAndBytes)
114
+ * Adapter merging: Yes (Full merged model)
115
+ * Language: Italian
116
+ * Domain: Medical β€” Cardiologia
117
+ * Training size: ~10,000 samples
118
+
119
+ ---
120
+
121
+ ## πŸ“œ License
122
+
123
+ This work is licensed under a
124
+ [Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License][cc-by-nc-nd].
125
+
126
+ [![CC BY-NC-ND 4.0][cc-by-nc-nd-image]][cc-by-nc-nd]
127
+
128
+ [cc-by-nc-nd]: http://creativecommons.org/licenses/by-nc-nd/4.0/
129
+ [cc-by-nc-nd-image]: https://licensebuttons.net/l/by-nc-nd/4.0/88x31.png
130
+ [cc-by-nc-nd-shield]: https://img.shields.io/badge/License-CC%20BY--NC--ND%204.0-lightgrey.svg
131
+
132
+ ---
133
+
134
+ ## 🀝 Acknowledgements
135
+
136
+ πŸ‘¨β€πŸ’» This project was developed by Mariano Barone, Roberta Di Marino, Francesco Di Serio, Giovanni Dioguardi, Marco Postiglione, Antonio Romano, Giuseppe Riccio, and Vincenzo Moscato at University of Naples, Federico II