Please correct config.json
#8
by Geonmo - opened
The presence of projection_dim is required but missing in the current implementation. Consequently, the utilization of CLIPVisionModelWithProjection and CLIPTextModelWithProjection is currently unavailable.
Geonmo changed pull request title from Update config.json to Please correct config.json
import torch
from transformers import CLIPVisionModelWithProjection
model_name = 'laion/CLIP-ViT-bigG-14-laion2B-39B-b160k'
clip_vision_model = CLIPVisionModelWithProjection.from_pretrained(model_name, torch_dtype=torch.float16)
then it will happen
`text_config_dict` is provided which will be used to initialize `CLIPTextConfig`. The value `text_config["id2label"]` will be overriden.
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:07<00:00, 3.51s/it]
RuntimeError: Error(s) in loading state_dict for CLIPVisionModelWithProjection:
size mismatch for visual_projection.weight: copying a param with shape torch.Size([1280, 1664]) from checkpoint, the shape in current model is torch.Size([512, 1664]).
You may consider adding `ignore_mismatched_sizes=True` in the model `from_pretrained` method.
fixed
rwightman changed pull request status to closed