clemsail commited on
Commit
3bfaeef
·
verified ·
1 Parent(s): 86df231

docs: add upstream base model official evaluations

Browse files
Files changed (1) hide show
  1. README.md +27 -0
README.md CHANGED
@@ -276,3 +276,30 @@ For reference benchmarks on the `gemma-4-E4B` base, see the
276
  Production usage: served via gateway alias `ailiance-apertus-<domain>` on
277
  <https://www.ailiance.fr> through the Apertus multi-LoRA hot-swap server
278
  (Studio :9322, 1 base + 10 LoRA dynamic swap, ~40GB VRAM).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
276
  Production usage: served via gateway alias `ailiance-apertus-<domain>` on
277
  <https://www.ailiance.fr> through the Apertus multi-LoRA hot-swap server
278
  (Studio :9322, 1 base + 10 LoRA dynamic swap, ~40GB VRAM).
279
+
280
+ ## Upstream base model — official evaluations
281
+
282
+ This LoRA fine-tunes [`swiss-ai/Apertus-70B-Instruct-2509`](https://huggingface.co/swiss-ai/Apertus-70B-Instruct-2509),
283
+ the EU-sovereign open-source LLM released by the Swiss AI Initiative. Below are
284
+ the **official scores** reported in the [Apertus Tech Report](https://arxiv.org/abs/2509.14233)
285
+ on a suite of multilingual reasoning benchmarks.
286
+
287
+ | Model | Avg | ARC | HellaSwag | WinoGrande | XNLI | XCOPA | PIQA |
288
+ |-----------------------------|------:|------:|----------:|-----------:|------:|------:|------:|
289
+ | **Apertus-70B** (this base) | 67.5 | 70.6 | 64.0 | 73.3 | 45.3 | 69.8 | 81.9 |
290
+ | Apertus-8B | 65.8 | 72.7 | 59.8 | 70.6 | 45.2 | 66.5 | 79.8 |
291
+ | Llama3.1-70B | 67.3 | 74.4 | 56.5 | 79.4 | 44.3 | 66.7 | 82.3 |
292
+ | Qwen2.5-72B | 69.8 | 76.2 | 67.5 | 78.0 | 46.9 | 68.2 | 82.0 |
293
+ | OLMo2-32B | 67.7 | 76.2 | 66.7 | 78.6 | 42.9 | 60.1 | 82.1 |
294
+ | EuroLLM-9B | 62.8 | 67.9 | 57.9 | 68.8 | 41.5 | 61.1 | 79.6 |
295
+
296
+ Many additional benchmark evaluations (pretraining/post-training phases,
297
+ multilingual in ~100 languages, long-context) are in Section 5 of the
298
+ [Apertus Tech Report](https://arxiv.org/abs/2509.14233).
299
+
300
+ **Source:** [official Apertus-70B-Instruct-2509 model card](https://huggingface.co/swiss-ai/Apertus-70B-Instruct-2509).
301
+
302
+ > **Reading these alongside this LoRA:** Apertus-70B is EU AI Act-compliant
303
+ > (`Apertus_EU_Code_of_Practice.pdf`, `Apertus_EU_Public_Summary.pdf` included
304
+ > in upstream weights). This LoRA inherits that compliance plus the
305
+ > general-capability floor shown above, then adds domain specialization.