clemsail commited on
Commit
77f0fd6
·
verified ·
1 Parent(s): 88bd3d2

docs: add upstream base model official evaluations

Browse files
Files changed (1) hide show
  1. README.md +27 -0
README.md CHANGED
@@ -126,3 +126,30 @@ See the full [Ailiance-fr LoRA collection](https://huggingface.co/Ailiance-fr).
126
  Production usage: served via gateway alias `ailiance-apertus-<domain>` on
127
  <https://www.ailiance.fr> through the Apertus multi-LoRA hot-swap server
128
  (Studio :9322, 1 base + 10 LoRA dynamic swap, ~40GB VRAM).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
126
  Production usage: served via gateway alias `ailiance-apertus-<domain>` on
127
  <https://www.ailiance.fr> through the Apertus multi-LoRA hot-swap server
128
  (Studio :9322, 1 base + 10 LoRA dynamic swap, ~40GB VRAM).
129
+
130
+ ## Upstream base model — official evaluations
131
+
132
+ This LoRA fine-tunes [`swiss-ai/Apertus-70B-Instruct-2509`](https://huggingface.co/swiss-ai/Apertus-70B-Instruct-2509),
133
+ the EU-sovereign open-source LLM released by the Swiss AI Initiative. Below are
134
+ the **official scores** reported in the [Apertus Tech Report](https://arxiv.org/abs/2509.14233)
135
+ on a suite of multilingual reasoning benchmarks.
136
+
137
+ | Model | Avg | ARC | HellaSwag | WinoGrande | XNLI | XCOPA | PIQA |
138
+ |-----------------------------|------:|------:|----------:|-----------:|------:|------:|------:|
139
+ | **Apertus-70B** (this base) | 67.5 | 70.6 | 64.0 | 73.3 | 45.3 | 69.8 | 81.9 |
140
+ | Apertus-8B | 65.8 | 72.7 | 59.8 | 70.6 | 45.2 | 66.5 | 79.8 |
141
+ | Llama3.1-70B | 67.3 | 74.4 | 56.5 | 79.4 | 44.3 | 66.7 | 82.3 |
142
+ | Qwen2.5-72B | 69.8 | 76.2 | 67.5 | 78.0 | 46.9 | 68.2 | 82.0 |
143
+ | OLMo2-32B | 67.7 | 76.2 | 66.7 | 78.6 | 42.9 | 60.1 | 82.1 |
144
+ | EuroLLM-9B | 62.8 | 67.9 | 57.9 | 68.8 | 41.5 | 61.1 | 79.6 |
145
+
146
+ Many additional benchmark evaluations (pretraining/post-training phases,
147
+ multilingual in ~100 languages, long-context) are in Section 5 of the
148
+ [Apertus Tech Report](https://arxiv.org/abs/2509.14233).
149
+
150
+ **Source:** [official Apertus-70B-Instruct-2509 model card](https://huggingface.co/swiss-ai/Apertus-70B-Instruct-2509).
151
+
152
+ > **Reading these alongside this LoRA:** Apertus-70B is EU AI Act-compliant
153
+ > (`Apertus_EU_Code_of_Practice.pdf`, `Apertus_EU_Public_Summary.pdf` included
154
+ > in upstream weights). This LoRA inherits that compliance plus the
155
+ > general-capability floor shown above, then adds domain specialization.