Quantized Models (GGUF, IQ, Imatrix)
Collection
Various GGUF quantizations of small models. Models with a "checkmark" are personal favorites. An "orange arrow" means it's being uploaded. • 97 items • Updated • 71
Model name:
MN-12B-Lyra-v4
Brief description:
A finetune of Mistral Nemo by Sao10K.
Uses the ChatML prompt format.
Presets:
You can use the built in ChatML presets within SillyTavern and adjust from there.
Alternatively, check out Virt-io's ChatML v1.9 presets here, make sure you read the repository page for how to use them properly.
Request page:
https://huggingface.co/Lewdiculous/Model-Requests/discussions/75Model link:
https://huggingface.co/Sao10K/MN-12B-Lyra-v4Quantized with llama.cpp:
b3707
- Downloads last month
- 8,030
Hardware compatibility
Log In to add your hardware
2-bit
3-bit
4-bit
5-bit
6-bit
8-bit
16-bit
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for Lewdiculous/MN-12B-Lyra-v4-GGUF-IQ-Imatrix
Base model
Sao10K/MN-12B-Lyra-v4