mgoin
/

Meta-Llama-3-70B-Instruct-Marlin

Text Generation

4-bit precision

Model card Files Files and versions

What conversion process are you using?

#2

by matt-psaltis-devbricks - opened May 23, 2024

matt-psaltis-devbricks

Curious which approach you're using to convert these models to Marlin? is it a published process?

Cheers!

Owner May 23, 2024

Hi @matt-psaltis-devbricks yep I used this script https://github.com/robertgshaw2-neuralmagic/marlin-example/blob/master/apply_gptq_save_marlin.py

matt-psaltis-devbricks

Awesome! Cheers for that

matt-psaltis-devbricks changed discussion status to closed May 23, 2024

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment