docs: add upstream base model official evaluations
Browse files
README.md
CHANGED
|
@@ -126,3 +126,30 @@ See the full [Ailiance-fr LoRA collection](https://huggingface.co/Ailiance-fr).
|
|
| 126 |
Production usage: served via gateway alias `ailiance-apertus-<domain>` on
|
| 127 |
<https://www.ailiance.fr> through the Apertus multi-LoRA hot-swap server
|
| 128 |
(Studio :9322, 1 base + 10 LoRA dynamic swap, ~40GB VRAM).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 126 |
Production usage: served via gateway alias `ailiance-apertus-<domain>` on
|
| 127 |
<https://www.ailiance.fr> through the Apertus multi-LoRA hot-swap server
|
| 128 |
(Studio :9322, 1 base + 10 LoRA dynamic swap, ~40GB VRAM).
|
| 129 |
+
|
| 130 |
+
## Upstream base model — official evaluations
|
| 131 |
+
|
| 132 |
+
This LoRA fine-tunes [`swiss-ai/Apertus-70B-Instruct-2509`](https://huggingface.co/swiss-ai/Apertus-70B-Instruct-2509),
|
| 133 |
+
the EU-sovereign open-source LLM released by the Swiss AI Initiative. Below are
|
| 134 |
+
the **official scores** reported in the [Apertus Tech Report](https://arxiv.org/abs/2509.14233)
|
| 135 |
+
on a suite of multilingual reasoning benchmarks.
|
| 136 |
+
|
| 137 |
+
| Model | Avg | ARC | HellaSwag | WinoGrande | XNLI | XCOPA | PIQA |
|
| 138 |
+
|-----------------------------|------:|------:|----------:|-----------:|------:|------:|------:|
|
| 139 |
+
| **Apertus-70B** (this base) | 67.5 | 70.6 | 64.0 | 73.3 | 45.3 | 69.8 | 81.9 |
|
| 140 |
+
| Apertus-8B | 65.8 | 72.7 | 59.8 | 70.6 | 45.2 | 66.5 | 79.8 |
|
| 141 |
+
| Llama3.1-70B | 67.3 | 74.4 | 56.5 | 79.4 | 44.3 | 66.7 | 82.3 |
|
| 142 |
+
| Qwen2.5-72B | 69.8 | 76.2 | 67.5 | 78.0 | 46.9 | 68.2 | 82.0 |
|
| 143 |
+
| OLMo2-32B | 67.7 | 76.2 | 66.7 | 78.6 | 42.9 | 60.1 | 82.1 |
|
| 144 |
+
| EuroLLM-9B | 62.8 | 67.9 | 57.9 | 68.8 | 41.5 | 61.1 | 79.6 |
|
| 145 |
+
|
| 146 |
+
Many additional benchmark evaluations (pretraining/post-training phases,
|
| 147 |
+
multilingual in ~100 languages, long-context) are in Section 5 of the
|
| 148 |
+
[Apertus Tech Report](https://arxiv.org/abs/2509.14233).
|
| 149 |
+
|
| 150 |
+
**Source:** [official Apertus-70B-Instruct-2509 model card](https://huggingface.co/swiss-ai/Apertus-70B-Instruct-2509).
|
| 151 |
+
|
| 152 |
+
> **Reading these alongside this LoRA:** Apertus-70B is EU AI Act-compliant
|
| 153 |
+
> (`Apertus_EU_Code_of_Practice.pdf`, `Apertus_EU_Public_Summary.pdf` included
|
| 154 |
+
> in upstream weights). This LoRA inherits that compliance plus the
|
| 155 |
+
> general-capability floor shown above, then adds domain specialization.
|