ibm-granite

/

granite-4.0-h-1b-base

like 35

Follow

IBM Granite 4.23k

The file have a extraneous file named "model.safetensors.index.json" is invalid. It that confuses llama.cpp python script used for GGUF conversion as it references "part" files that do not exist since the weights are all in a single file in the repo. (i.e., "model.safetensors"). This leads to an os.read() error in HF Transformers "file not found". This may have accidentally worked for older HF Transformers versions at the time this model was published; however, on 4.57.1 and newer the logic has changed in Transformers where this read() error is occurring.

Please note that this file does not appear (as it is invalid) in other Granite 4 repos. which use a single weights file; it only appear for:

• "granite-4.0-1b-base" or
• "granite-4.0-h-1b-base"

I have confirmed that once the file "model.safetensors.index.json" is deleted before calling the python GGUF conversion script in llama.cpp (which uses HF Transformers) will work without error.