Update README.md
Browse files
README.md
CHANGED
|
@@ -45,4 +45,21 @@ scores = model.predict(query_documents)
|
|
| 45 |
print(scores)
|
| 46 |
```
|
| 47 |
|
| 48 |
-
The model can also be inferenced using ZeroEntropy's [/models/rerank](https://docs.zeroentropy.dev/api-reference/models/rerank) endpoint.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 45 |
print(scores)
|
| 46 |
```
|
| 47 |
|
| 48 |
+
The model can also be inferenced using ZeroEntropy's [/models/rerank](https://docs.zeroentropy.dev/api-reference/models/rerank) endpoint, and on [AWS Marketplace](https://aws.amazon.com/marketplace/pp/prodview-o7avk66msiukc).
|
| 49 |
+
|
| 50 |
+
## Evaluations
|
| 51 |
+
|
| 52 |
+
NDCG@10 scores between `zerank-2` and competing closed-source proprietary rerankers. Since we are evaluating rerankers, OpenAI's `text-embedding-3-small` is used as an initial retriever for the Top 100 candidate documents.
|
| 53 |
+
|
| 54 |
+
| Domain | OpenAI embeddings | ZeroEntropy zerank-2 | ZeroEntropy zerank-1 | Gemini 2.5 Flash (Listwise) | Cohere rerank-3.5 |
|
| 55 |
+
|------------------|-------------------|----------------------|----------------------|-----------------------------|-------------------|
|
| 56 |
+
| Web | 0.3819 | **0.6346** | 0.6069 | 0.5765 | 0.5594 |
|
| 57 |
+
| Conversational | 0.4305 | **0.6140** | 0.5801 | 0.6021 | 0.5648 |
|
| 58 |
+
| STEM & Logic | 0.3744 | **0.6521** | 0.6283 | 0.5447 | 0.5418 |
|
| 59 |
+
| Code | 0.4582 | **0.6528** | 0.6310 | 0.6128 | 0.5364 |
|
| 60 |
+
| Legal | 0.4101 | **0.6644** | 0.6222 | 0.5565 | 0.5257 |
|
| 61 |
+
| Biomedical | 0.4783 | **0.7217** | 0.6967 | 0.5371 | 0.6246 |
|
| 62 |
+
| Finance | 0.6232 | 0.7600 | 0.7539 | **0.7694** | 0.7402 |
|
| 63 |
+
| **Average** | **0.4509** | **0.6714** | **0.6456** | **0.5999** | **0.5847** |
|
| 64 |
+
|
| 65 |
+
<img src="https://cdn-uploads.huggingface.co/production/uploads/65ec60ccfc59f6e77ecc9ccb/UiDp8LsY4XIdRK5i3CAdD.png" alt="Graph showing the same table" width="1000"/>
|