Flipped accuracy table for better visualization on HF
Browse files
README.md
CHANGED
|
@@ -125,10 +125,23 @@ Latency measured with TensorRT + `trtexec`, GPU compute time only
|
|
| 125 |
Evaluated on the sts17 validation split. The quantized model
|
| 126 |
retains nearly all of the FP32 accuracy with a small tolerance.
|
| 127 |
|
| 128 |
-
|
|
| 129 |
-
|---|---|---|---|
|
| 130 |
-
|
|
| 131 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 132 |
|
| 133 |
## Creating Your Own Optimized Models
|
| 134 |
|
|
|
|
| 125 |
Evaluated on the sts17 validation split. The quantized model
|
| 126 |
retains nearly all of the FP32 accuracy with a small tolerance.
|
| 127 |
|
| 128 |
+
| Metric | FP32 (ours) | **Embedl Paraphrase Multilingual Minilm L12 V2 INT8** | Δ |
|
| 129 |
+
|---|---|---|---|
|
| 130 |
+
| Spearman ρ | 0.8130 | **0.8008** | -0.0122 |
|
| 131 |
+
| ρ (ar-ar) | 0.7915 | **0.7906** | -0.0010 |
|
| 132 |
+
| ρ (default) | 0.7970 | **0.7868** | -0.0102 |
|
| 133 |
+
| ρ (en-ar) | 0.8122 | **0.7914** | -0.0208 |
|
| 134 |
+
| ρ (en-de) | 0.8422 | **0.8215** | -0.0207 |
|
| 135 |
+
| ρ (en-en) | 0.8687 | **0.8638** | -0.0049 |
|
| 136 |
+
| ρ (en-tr) | 0.7674 | **0.7555** | -0.0119 |
|
| 137 |
+
| ρ (es-en) | 0.8444 | **0.8300** | -0.0143 |
|
| 138 |
+
| ρ (es-es) | 0.8556 | **0.8328** | -0.0228 |
|
| 139 |
+
| ρ (fr-en) | 0.7659 | **0.7536** | -0.0123 |
|
| 140 |
+
| ρ (it-en) | 0.8235 | **0.8148** | -0.0087 |
|
| 141 |
+
| ρ (ko-ko) | 0.7703 | **0.7628** | -0.0075 |
|
| 142 |
+
| ρ (nl-en) | 0.8171 | **0.8059** | -0.0112 |
|
| 143 |
+
|
| 144 |
+
FP32 baseline: [`sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2`](https://huggingface.co/sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2).
|
| 145 |
|
| 146 |
## Creating Your Own Optimized Models
|
| 147 |
|