Vision support
#6
by raulalonsoctic - opened
Any chance you could add the vision decoder to this model? This is the only NVFP4 quantization that works reliably under vLLM
raulalonsoctic changed discussion status to closed
raulalonsoctic changed discussion status to open
The vision encoder is in the model in shard 5, it is unquantized (therefore in ignore list) and it should work.
I have uploaded a new revision that includes some configs that were missing for multi-modal purposes. Can you try with that and see if vision works? Thanks.
Working great on my first tests. Thank you very much!!!
Thanks, Im uploading the nvfp4 for the 122B version as we speak, with all configs etc.