nielsr HF Staff commited on
Commit
6abc905
·
verified ·
1 Parent(s): 7713b2c

Add model card for HAC

Browse files

This PR adds a comprehensive model card for the HAC model. It includes:
- A link to the paper: [HAC: Parameter-Efficient Hyperbolic Adaptation of CLIP for Zero-Shot VQA](https://huggingface.co/papers/2604.23665).
- A link to the official GitHub repository.
- Relevant metadata including the `image-text-to-text` pipeline tag.
- Instructions for environment setup and evaluation as found in the repository.

Files changed (1) hide show
  1. README.md +51 -0
README.md ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ pipeline_tag: image-text-to-text
3
+ ---
4
+
5
+ # HAC: Parameter-Efficient Hyperbolic Adaptation of CLIP for Zero-Shot VQA
6
+
7
+ HAC (Hyperbolic Adaptation of CLIP) is a parameter-efficient framework that enables pretrained CLIP models to transition into hyperbolic space via lightweight fine-tuning. This approach captures hierarchical structures more effectively than traditional Euclidean embeddings, specifically for tasks like zero-shot Visual Question Answering (VQA).
8
+
9
+ This repository contains the weights for **HAC-B w/ LoRA**.
10
+
11
+ - **Paper:** [HAC: Parameter-Efficient Hyperbolic Adaptation of CLIP for Zero-Shot VQA](https://huggingface.co/papers/2604.23665)
12
+ - **GitHub Repository:** [https://github.com/fdibiton/HAC](https://github.com/fdibiton/HAC)
13
+
14
+ ## Environment Setup
15
+
16
+ Create and configure the environment using Conda:
17
+
18
+ ```bash
19
+ git clone https://github.com/fdibiton/HAC.git
20
+ cd HAC
21
+ conda create -n hac python=3.9 --yes
22
+ conda activate hac
23
+
24
+ # Install dependencies
25
+ python -m pip install --pre timm
26
+ python -m pip install -r requirements.txt
27
+ ```
28
+
29
+ ## Evaluation
30
+
31
+ To run zero-shot VQA evaluation with the HAC-B w/ LoRA model, place the `hac_vit_b_lora.pth` file in the `./checkpoints` directory and run:
32
+
33
+ ```bash
34
+ python scripts/evaluate.py \
35
+ --config configs/eval_vqa_all_categories.py \
36
+ --train-config configs/train_hac_vit_b_lora.py \
37
+ --checkpoint-path checkpoints/hac_vit_b_lora.pth
38
+ ```
39
+
40
+ > **Note:** The VQA evaluation datasets need to be downloaded and arranged beforehand. Please refer to the instructions in the [GitHub repository](https://github.com/fdibiton/HAC) for details.
41
+
42
+ ## Citation
43
+
44
+ ```bibtex
45
+ @inproceedings{dibiton2026hac,
46
+ title={HAC: Parameter-Efficient Hyperbolic Adaptation of CLIP for Zero-Shot VQA},
47
+ author={Dibitonto, Francesco and Beyan, Cigdem and Murino, Vittorio},
48
+ booktitle={International Conference on Pattern Recognition (ICPR)},
49
+ year={2026}
50
+ }
51
+ ```