Question about enc=Qwen3-Embedding-0.6B-lora-dec=Qwen3-0.6B-lora
#2
by dongjiancheng77 - opened
Hello, very great work! I noticed you have a checkpoint: enc=Qwen3-Embedding-0.6B-lora-dec=Qwen3-0.6B-lora.I was wondering how you use Qwen3-Embedding-0.6B? Is it used exactly the same way as Qwen3-0.6B? And how is the performance?
Qwen3-Embedding-0.6B has the same model architecture as Qwen3-0.6B, the only differences are:
- Qwen3-Embedding-0.6B's tokenizer automatically add an <|endoftext|> at the end of the text
- Qwen3-Embedding-0.6B doesn't have lm_head
In our method, for the encoder, lm_head is not used, so we use Qwen3-Embedding-0.6B exactly the same way as Qwen3-0.6B. Its performance is slightly better than Qwen3-0.6B, maybe because it is further trained for encoding a document.
Thanks for the clear explanation!