---
license: apache-2.0
tags:
- OmniQuant
- Quantization
- Weight-Quantization
---

This repo contains all the required Learnable Weight Clipping for Omniquant https://arxiv.org/abs/2308.13137. 

# How to use it?

To use them please first run:

```
!mkdir OmniQuant_LWC
!git lfs install
!git clone https://huggingface.co/Tfloow/OmniQuant
!cp Omniquant/* OmniQuant_LWC/ # To avoid conflict in names
!git clone https://github.com/OpenGVLab/OmniQuant/tree/main
```

Then you can run OmniQuant as usual with the flag `--resume`:

```
CUDA_VISIBLE_DEVICES=0 python OmniQuant/main.py \
--model NAME_OF_MODEL  \
--epochs 0 --output_dir ./log/test \
--eval_ppl --wbits 4 --abits 16 --group_size 128 --lwc \
--resume OmniQuant_LWC/NAME_OF_MODEL-w4a16g128.pth
```

# Methodology

The weights were run using a fork of OmniQuant available at [calibration.ipynb](https://github.com/Tfloow/AndaQuant/blob/main/calibration.ipynb)