docs: add upstream base model official evaluations
Browse files
README.md
CHANGED
|
@@ -276,3 +276,30 @@ For reference benchmarks on the `gemma-4-E4B` base, see the
|
|
| 276 |
Production usage: served via gateway alias `ailiance-apertus-<domain>` on
|
| 277 |
<https://www.ailiance.fr> through the Apertus multi-LoRA hot-swap server
|
| 278 |
(Studio :9322, 1 base + 10 LoRA dynamic swap, ~40GB VRAM).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 276 |
Production usage: served via gateway alias `ailiance-apertus-<domain>` on
|
| 277 |
<https://www.ailiance.fr> through the Apertus multi-LoRA hot-swap server
|
| 278 |
(Studio :9322, 1 base + 10 LoRA dynamic swap, ~40GB VRAM).
|
| 279 |
+
|
| 280 |
+
## Upstream base model — official evaluations
|
| 281 |
+
|
| 282 |
+
This LoRA fine-tunes [`swiss-ai/Apertus-70B-Instruct-2509`](https://huggingface.co/swiss-ai/Apertus-70B-Instruct-2509),
|
| 283 |
+
the EU-sovereign open-source LLM released by the Swiss AI Initiative. Below are
|
| 284 |
+
the **official scores** reported in the [Apertus Tech Report](https://arxiv.org/abs/2509.14233)
|
| 285 |
+
on a suite of multilingual reasoning benchmarks.
|
| 286 |
+
|
| 287 |
+
| Model | Avg | ARC | HellaSwag | WinoGrande | XNLI | XCOPA | PIQA |
|
| 288 |
+
|-----------------------------|------:|------:|----------:|-----------:|------:|------:|------:|
|
| 289 |
+
| **Apertus-70B** (this base) | 67.5 | 70.6 | 64.0 | 73.3 | 45.3 | 69.8 | 81.9 |
|
| 290 |
+
| Apertus-8B | 65.8 | 72.7 | 59.8 | 70.6 | 45.2 | 66.5 | 79.8 |
|
| 291 |
+
| Llama3.1-70B | 67.3 | 74.4 | 56.5 | 79.4 | 44.3 | 66.7 | 82.3 |
|
| 292 |
+
| Qwen2.5-72B | 69.8 | 76.2 | 67.5 | 78.0 | 46.9 | 68.2 | 82.0 |
|
| 293 |
+
| OLMo2-32B | 67.7 | 76.2 | 66.7 | 78.6 | 42.9 | 60.1 | 82.1 |
|
| 294 |
+
| EuroLLM-9B | 62.8 | 67.9 | 57.9 | 68.8 | 41.5 | 61.1 | 79.6 |
|
| 295 |
+
|
| 296 |
+
Many additional benchmark evaluations (pretraining/post-training phases,
|
| 297 |
+
multilingual in ~100 languages, long-context) are in Section 5 of the
|
| 298 |
+
[Apertus Tech Report](https://arxiv.org/abs/2509.14233).
|
| 299 |
+
|
| 300 |
+
**Source:** [official Apertus-70B-Instruct-2509 model card](https://huggingface.co/swiss-ai/Apertus-70B-Instruct-2509).
|
| 301 |
+
|
| 302 |
+
> **Reading these alongside this LoRA:** Apertus-70B is EU AI Act-compliant
|
| 303 |
+
> (`Apertus_EU_Code_of_Practice.pdf`, `Apertus_EU_Public_Summary.pdf` included
|
| 304 |
+
> in upstream weights). This LoRA inherits that compliance plus the
|
| 305 |
+
> general-capability floor shown above, then adds domain specialization.
|