config.json error

by martoon - opened Mar 20, 2025

Mar 20, 2025

Hello,
I am sorry, maybe I did not get something, I don't suceed in load this model with the code

Load model directly

from transformers import AutoModel
model = AutoModel.from_pretrained("jpacifico/Chocolatine-2-14B-Instruct-v2.0.3-Q8_0-GGUF")

error:
ValueError: Unrecognized model in jpacifico/Chocolatine-2-14B-Instruct-v2.0.3-Q8_0-GGUF. Should have a model_type key in its config.json

the not quantized model loads fine

jpacifico

Owner Mar 20, 2025

Hi @martoon I’d say that with transformers you cannot directly load a GGUF-quantized model (.gguf) using AutoModel.from_pretrained(), because GGUF is a format specific to llama.cpp and is not natively supported by transformers.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment