Add model card

README.md

@@ -0,0 +1,114 @@
+---
+license: apache-2.0
+base_model: Qwen/Qwen3-Reranker-4B
+base_model_relation: quantized
+library_name: openvino
+tags:
+  - openvino
+  - openvino-export
+  - text-ranking
+  - reranker
+  - qwen3
+language:
+  - multilingual
+pipeline_tag: text-ranking
+---
+
+# Qwen3-Reranker-4B — OpenVINO IR (INT8 weight-only, asymmetric)
+
+> **This is a redistribution.** For the model's intended use, instruction
+> format, full evaluation (MTEB-R / CMTEB-R / MMTEB-R / MLDR / MTEB-Code /
+> FollowIR), and citation, please see the upstream card:
+> **[Qwen/Qwen3-Reranker-4B](https://huggingface.co/Qwen/Qwen3-Reranker-4B)**.
+
+OpenVINO IR conversion of
+[Qwen/Qwen3-Reranker-4B](https://huggingface.co/Qwen/Qwen3-Reranker-4B),
+weight-only quantized to **INT8 asymmetric** via NNCF. Intended for the
+[OpenArc](https://github.com/SearchSavior/OpenArc) reranker engine and
+`optimum-intel` pipelines targeting Intel CPUs / iGPUs / dGPUs / NPUs.
+
+## Files
+
+- `openvino_model.{xml,bin}` — Qwen3 (4B) decoder, INT8 weights (~4.0 GB)
+- `openvino_tokenizer.{xml,bin}` / `openvino_detokenizer.{xml,bin}` — OpenVINO Tokenizers IR
+- `chat_template.jinja`, `generation_config.json`
+- Standard HF tokenizer files: `tokenizer.json`, `tokenizer_config.json`,
+  `special_tokens_map.json`, `vocab.json`, `merges.txt`
+- `LICENSE`, `NOTICE` — Apache-2.0 with attribution to the upstream Qwen Team.
+
+## Architecture
+
+| | |
+|---|---|
+| Base model | Qwen3 ForCausalLM (Qwen3-4B-Base) |
+| Hidden size | 2560 |
+| Layers | 36 |
+| Attention heads / KV heads | 32 / 8 |
+| Max position | 40 960 |
+| Vocabulary | 151 669 |
+| Source dtype | bfloat16 |
+| Quantization | NNCF INT8 weight-only, asymmetric |
+
+## Usage with OpenArc
+
+```bash
+openarc add qwen3-4b-reranker \
+  --model-path /path/to/Qwen3-Reranker-4B-int8-ov \
+  --model-type rerank \
+  --engine optimum \
+  --device GPU
+
+openarc serve
+# POST /v1/rerank  {"model": "qwen3-4b-reranker", "query": "...", "documents": [...]}
+```
+
+## Conversion notes
+
+The standard CLI route currently fails silently for this model on
+`optimum-intel @ HEAD`:
+
+```bash
+optimum-cli export openvino --weight-format int8 \
+  --model Qwen/Qwen3-Reranker-4B ./out
+# exits 0; openvino_model.xml is 0 bytes, openvino_model.bin is a ~13 MB stub
+```
+
+The Python API path produces a usable model:
+
+```python
+from optimum.intel import OVModelForCausalLM, OVWeightQuantizationConfig
+
+quant = OVWeightQuantizationConfig(bits=8, sym=False)
+m = OVModelForCausalLM.from_pretrained(
+    "Qwen/Qwen3-Reranker-4B",
+    export=True,
+    quantization_config=quant,
+    trust_remote_code=True,
+)
+m.save_pretrained("./out")
+```
+
+Tokenizer / detokenizer IR generated separately via
+`openvino_tokenizers.convert_tokenizer(..., with_detokenizer=True)`.
+
+A more detailed walkthrough lives in
+[OpenArc/docs/openvino_qwen3.md](https://github.com/SearchSavior/OpenArc).
+
+## License
+
+Apache-2.0, inherited from
+[Qwen/Qwen3-Reranker-4B](https://huggingface.co/Qwen/Qwen3-Reranker-4B).
+See `LICENSE` and `NOTICE` in this repo.
+
+## Citation
+
+From the upstream model card:
+
+```bibtex
+@article{qwen3embedding,
+  title={Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models},
+  author={Zhang, Yanzhao and Li, Mingxin and Long, Dingkun and Zhang, Xin and Lin, Huan and Yang, Baosong and Xie, Pengjun and Yang, An and Liu, Dayiheng and Lin, Junyang and Huang, Fei and Zhou, Jingren},
+  journal={arXiv preprint arXiv:2506.05176},
+  year={2025}
+}
+```