Tfloow/OmniQuant

This repo contains all the required Learnable Weight Clipping for Omniquant https://arxiv.org/abs/2308.13137.

How to use it?

To use them please first run:

!mkdir OmniQuant_LWC
!git lfs install
!git clone https://huggingface.co/Tfloow/OmniQuant
!cp Omniquant/* OmniQuant_LWC/ # To avoid conflict in names
!git clone https://github.com/OpenGVLab/OmniQuant/tree/main

Then you can run OmniQuant as usual with the flag --resume:

CUDA_VISIBLE_DEVICES=0 python OmniQuant/main.py \
--model NAME_OF_MODEL  \
--epochs 0 --output_dir ./log/test \
--eval_ppl --wbits 4 --abits 16 --group_size 128 --lwc \
--resume OmniQuant_LWC/NAME_OF_MODEL-w4a16g128.pth

Methodology

The weights were run using a fork of OmniQuant available at calibration.ipynb