Upload README.md with huggingface_hub

README.md

@@ -13,38 +13,49 @@ tags:
 pipeline_tag: token-classification
 ---
 
-# GLiNER2 Multi-v1 (ONNX Fragmented)
+# GLiNER2 Multi-v1 (ONNX Fragmented & IOBinding)
 
 This repository contains the ONNX-exported weights for **GLiNER2-Multi-v1**.
 The model is specifically exported in a fragmented format (encoder, span_rep, count_pred, count_lstm, classifier) to be directly compatible with [gliner2-rs](https://github.com/SemplificaAI/gliner2-rs), the official Zero-Python Native Rust Inference Engine for GLiNER2.
 
+### 🆕 Update: V2 Zero-Copy IOBinding Models Available!
+We have introduced **V2 fused models** (`fp16_v2` and `fp32_v2`) that fuse `Gather`, `ArgMax`, and `MatMul` operations directly into the ONNX graph. By using ORT's `IoBinding`, these models ensure that tensors **never leave the GPU/NPU VRAM**, completely bypassing the PCIe bus and reducing inference latency by ~30% on discrete GPUs.
+
+## 📂 Available Variants
+
+*   **`fp16_v2`** *(Recommended)*: Zero-Copy VRAM optimized models. Fused operations with Full IO Types (native FP16). Drastically reduces inference time on NVIDIA CUDA, AMD ROCm, and Apple CoreML. Requires `gliner2-rs >= 0.4.1`.
+*   **`fp32_v2`**: High precision V2 fusions for CPU execution (AVX, XNNPACK). Requires `gliner2-rs >= 0.4.1`.
+*   **`fp16`** *(Standard)*: Legacy Float16 ONNX models. Slower on discrete GPUs due to PCIe transfers, but completely stable and supported everywhere.
+*   **`fp32`** *(Standard)*: Legacy Float32 ONNX models.
+
 ## 🚀 Performance & Benchmarks
 
 The ONNX conversion, combined with the Rust native engine (`ort` binding), allows this model to run extremely fast on both GPUs and edge devices like NPUs.
 
 **Benchmark Task:** Tested on complex text extraction tasks spanning up to 62 classes (metrics normalized per extracted entity to allow cross-device comparison).
 
-| Hardware | Execution Provider | Avg Time / Entity |
-| :--- | :--- | :--- |
-| **NVIDIA RTX 4090** | CUDA (FP16) | **~12.0 ms** 🚀 |
-| **NVIDIA RTX 3090** | CUDA (FP16) | **~11.6 ms** 🚀 |
-| **Qualcomm Snapdragon X Elite** | QNN (NPU Native) | **~22.78 ms** ✨ |
-| **Qualcomm Snapdragon X Elite** | CPU (ARM NEON) | **~28.62 ms** |
-| **AMD Ryzen 9 5900XT** (16-Core) | CPU (x86 AVX2) | **~30.16 ms** 💻 |
-
-*Note: The NPU matches high-end Desktop CPUs while consuming a fraction of the power!*
+| Hardware | Execution Provider | Model Variant | Avg Time / Entity |
+| :--- | :--- | :--- | :--- |
+| **NVIDIA RTX 4090** | CUDA (V2 IOBinding) | `fp16_v2` | **~7.0 ms** ⚡ |
+| **NVIDIA RTX 3090** | CUDA (V2 IOBinding) | `fp16_v2` | **~7.2 ms** ⚡ |
+| **NVIDIA RTX 4090** | CUDA (V1 Standard) | `fp16` | **~12.0 ms** 🚀 |
+| **NVIDIA RTX 3090** | CUDA (V1 Standard) | `fp16` | **~11.6 ms** 🚀 |
+| **Qualcomm Snapdragon X Elite** | QNN (NPU Native) | `fp16` | **~22.78 ms** ✨ |
+| **AMD Ryzen 9 5900XT** (16-Core) | CPU (x86 AVX2) | `fp32_v2` | **~20.6 ms** 💻 |
+| **Qualcomm Snapdragon X Elite** | CPU (ARM NEON) | `fp32` | **~28.62 ms** |
 
 ## 📦 Usage in Rust
 
-You can dynamically download and execute these ONNX weights from Rust in 3 lines of code:
+You can dynamically download and execute these ONNX weights from Rust in 3 lines of code. With `gliner2-rs >= 0.4.1`, the engine automatically detects if you are using V1 or V2 models and routes the execution perfectly.
 
 ```rust
 use gliner2_inference::{Gliner2Engine, ModelType, SchemaTask};
 
-// Auto-downloads the FP16 models from this HuggingFace repo
+// Auto-downloads the V2 FP16 models from this HuggingFace repo
+// and automatically switches to the high-performance IOBinding Engine!
 let engine = Gliner2Engine::from_pretrained(
     "SemplificaAI/gliner2-multi-v1-onnx",
-    Some("fp16"),
+    Some("fp16_v2"),
     ModelType::HuggingFace
 )?;
 
@@ -60,3 +71,4 @@ let (entities, _, _) = engine.extract(text, &tasks)?;
 
 ## 🔧 Model Fixes
 - `count_lstm` has been successfully exported with dynamic sequence unrolling by replacing the training `CompileSafeGRU` with native `nn.GRU`, resolving Out-of-Bounds `Gather` ONNX errors for variable length texts.
+- **(V2)** `Scorer` now uses a heavily optimized fused combination of `Reshape` + `MatMul` + `Transpose` instead of `Einsum`, ensuring compatibility with execution providers that don't support `Einsum` in FP16 (e.g. QNN, CoreML).