Dequanting Script
#1
by RonanMcGovern - opened
Would you have a script you could share for dequantizing? I assume from nf4?
Hi @RonanMcGovern , you can refer to https://github.com/ludwig-ai/ludwig/blob/master/examples/llm_base_model_dequantization/phi_2_dequantization.py for an example of a script that performs dequantization on a model.
many thanks
RonanMcGovern changed discussion status to closed