ExLlamav3_HF AttributeError: 'dict' object has no attribute 'to_dict'

by Gesard - opened Mar 10

Mar 10

Thanks for the quaints as usual turbo. I was wondering if you or anyone has issues loading these with the HF pipeline on oobabooga. I get an AttributeError, 'dict' object has no attribute 'to_dict' whenever I try to load. Not a big deal as I'm able to load it just fine with ExLlamav3, but I get lower VRAM usage when using HF.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment