mmjproj files
Hello,
Is the mmproj file the same as for the non-imatrix version?
If so, would it be possible to have them in this repo as well, for compatibility with tools with auto download capability such as LM Studio?
Is the mmproj file the same as for the non-imatrix version?
The mmproj is always the same for all quants not matter if static or weighted/imatrix quants. You are supposed to just download them from the static repository. The model card is quite garbage as we never completed the model card update project. I don't think with the current model card it is possible for someone to find the mmproj files without somehow knowing that he needs to search in the static repo. This really needs to be improved as the amount of vision models increased by a lot recently.
If so, would it be possible to have them in this repo as well, for compatibility with tools with auto download capability such as LM Studio?
I wasn't aware of tools downloading them automatically. I'm only using llama.cpp and there I have to download them manually. They are the same exact files so just uploading twice seems a bit stupid but if many tools download them automatically from the same repository it might be worth it as they are relatively small. @mradermacher What is your opinion on this?
The model card is on my mind almost every day...
I think the auto-download should be improved. Even if it's not wasted space per se, I think it's conceptually wrong to just duplicate files because some tool is too dumb. I think those auto-downloads tools are obviously not very well designed if they can't get that right. And LM studio has a good track record of failing it's downloads anyway, with or without mmproj. I am not dead set on this, but I think the tools should make an effort first - there are far fewer tools than there are repositories.
there should be some kind of metadata on this repo that refer the other repo so that the discovery can be made. This model is arguably useless without mmproj tbh. As for saving space, if it is the same exact file would xet be able to deduplicate it across repos? I guess on HF there are a lot of duplicated files so it would make sense for them to have such a functionality.
Well, every model card links to the sttaic quants (but does not say that the mmproj files are there). The model cards will be changed at some point, though, it's a TODO for almost a year now.
I don't know if they can do that, but I give it a high chance. They could have done that without xet as well, though. The problem is keeping track of whether the chunks (for xet, filkes for lfs) are still referenced.
My primary objection is not wasted physical space but conceptional space, and the added management and danger of things getting out of sync.
Arguably, if the existing model card would simply have a row with links the static repo, the problem already would not exist. The new model cards will have a link to our own metadata page (e.g. https://hf.tst.eu/model#Nanonets-OCR-s-i1-GGUF), which lists them already, but adding that link is part of that TODO. And at the moment, I am more swamped with issues than ever before, and physically and even emotionally drained the remaining time. Ah well, I just have to find free time and leisure for it :)