| --- |
| license: apache-2.0 |
| base_model: swiss-ai/Apertus-70B-Instruct-2509 |
| library_name: peft |
| tags: |
| - mlx |
| - lora |
| - peft |
| - ailiance |
| - apertus |
| - math |
| language: |
| - en |
| - fr |
| pipeline_tag: text-generation |
| --- |
| |
| # Ailiance — Apertus-70B-Instruct math LoRA |
|
|
| LoRA adapter fine-tuned on `swiss-ai/Apertus-70B-Instruct-2509` for **math** tasks. |
|
|
| > Maintained by **Ailiance** — French AI org publishing EU AI Act aligned LoRA adapters and datasets. |
|
|
| ## Quick start (MLX) |
|
|
| ```python |
| from mlx_lm import load, generate |
| |
| model, tokenizer = load( |
| "swiss-ai/Apertus-70B-Instruct-2509", |
| adapter_path="Ailiance-fr/apertus-math-lora", |
| ) |
| |
| print(generate(model, tokenizer, prompt="...")) |
| ``` |
|
|
| ## Training |
|
|
| | Hyperparameter | Value | |
| |------------------|------------------------| |
| | Base model | `swiss-ai/Apertus-70B-Instruct-2509` | |
| | Method | LoRA via `mlx-lm` | |
| | Rank | 16 | |
| | Scale | 2.0 | |
| | Alpha | 32 | |
| | Max seq length | 1024 | |
| | Iterations | 500 | |
| | Optimizer | Adam, LR 1e-5 | |
| | Hardware | Apple M3 Ultra 512 GB | |
|
|
| ## Training data lineage |
|
|
| Derived from the internal **eu-kiki / mascarade** curation. All upstream samples |
| are synthetic, permissively-licensed, or generated from Apache-2.0 base resources. |
| See the [Ailiance-fr catalog](https://huggingface.co/Ailiance-fr) for related cards. |
|
|
| ## Benchmark roadmap |
|
|
| This LoRA has **not yet been evaluated** through `electron-bench` (the current |
| pipeline supports `gemma-4-E4B` base only). Training was completed with the |
| standard `mlx-lm` LoRA trainer (rank 16, alpha 32, scale 2.0, AdamW |
| LR 1e-5, 500 iters) — full hyperparameters are in the `Training` table above. |
|
|
| Planned evaluations: |
|
|
| - Perplexity on the validation split of the training data |
| - Functional benchmark on **apertus**-specific tasks |
| - Comparison vs base `swiss-ai/Apertus-70B-Instruct-2509` |
|
|
| Track progress: [ailiance-bench issues](https://github.com/ailiance/ailiance-bench/issues). |
|
|
| For reference benchmarks on the `gemma-4-E4B` base, see the |
| [base-vs-LoRA matrix](https://github.com/ailiance/ailiance-bench/blob/main/bench-results/compare_base_vs_lora.md). |
|
|
| ## License chain |
|
|
| | Component | License | |
| |-----------------------------------|-------------------| |
| | Base model (`swiss-ai/Apertus-70B-Instruct-2509`) | apache-2.0 | |
| | Training data (internal Ailiance curation (synthetic + permissive sources)) | apache-2.0 | |
| | **LoRA adapter (this repo)** | **apache-2.0**| |
|
|
| _All upstream components are Apache 2.0 / MIT — LoRA inherits permissive terms._ |
|
|
| ## EU AI Act compliance |
|
|
| - **Article 53(1)(c)**: training data licenses preserved (per-dataset cards declare upstream licenses). |
| - **Article 53(1)(d)**: training data summary — see upstream dataset cards on Ailiance-fr. |
| - **GPAI Code of Practice (July 2025)**: base `swiss-ai/Apertus-70B-Instruct-2509` released under apache-2.0. |
| - **No web scraping by Ailiance**, **no licensed data**, **no PII**. |
| - Upstream Stack Exchange content (where applicable) is CC-BY-SA-4.0 and propagates to this adapter. |
|
|
| ## License |
|
|
| LoRA weights: **apache-2.0** — see License chain table above for derivation rationale. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{ailiance_apertus_math_2026, |
| author = {Ailiance}, |
| title = {Ailiance — Apertus-70B-Instruct math LoRA}, |
| year = {2026}, |
| publisher = {Hugging Face}, |
| url = {https://huggingface.co/Ailiance-fr/apertus-math-lora} |
| } |
| ``` |
|
|
| ## Related |
|
|
| See the full [Ailiance-fr LoRA collection](https://huggingface.co/Ailiance-fr). |
|
|
|
|
| ## Bench comparison (2026-05-11) |
|
|
| ### Base model (Apertus-70B-Instruct-2509) capability |
|
|
| | Task | Score | Notes | |
| |---|---:|---| |
| | ARC-Easy acc / acc_norm | **0.81 / 0.77** | W3 lm-eval-harness BF16 | |
| | GSM8K-CoT | TIMEOUT (1800s budget) | base 70B BF16 too slow for CoT | |
| | MMLU-Pro Computer Science | TIMEOUT | | |
| |
| ### This LoRA (tuned) — bench PENDING |
| |
| Production usage: served via gateway alias `ailiance-apertus-<domain>` on |
| <https://www.ailiance.fr> through the Apertus multi-LoRA hot-swap server |
| (Studio :9322, 1 base + 10 LoRA dynamic swap, ~40GB VRAM). |
| |
| ## Upstream base model — official evaluations |
| |
| This LoRA fine-tunes [`swiss-ai/Apertus-70B-Instruct-2509`](https://huggingface.co/swiss-ai/Apertus-70B-Instruct-2509), |
| the EU-sovereign open-source LLM released by the Swiss AI Initiative. Below are |
| the **official scores** reported in the [Apertus Tech Report](https://arxiv.org/abs/2509.14233) |
| on a suite of multilingual reasoning benchmarks. |
| |
| | Model | Avg | ARC | HellaSwag | WinoGrande | XNLI | XCOPA | PIQA | |
| |-----------------------------|------:|------:|----------:|-----------:|------:|------:|------:| |
| | **Apertus-70B** (this base) | 67.5 | 70.6 | 64.0 | 73.3 | 45.3 | 69.8 | 81.9 | |
| | Apertus-8B | 65.8 | 72.7 | 59.8 | 70.6 | 45.2 | 66.5 | 79.8 | |
| | Llama3.1-70B | 67.3 | 74.4 | 56.5 | 79.4 | 44.3 | 66.7 | 82.3 | |
| | Qwen2.5-72B | 69.8 | 76.2 | 67.5 | 78.0 | 46.9 | 68.2 | 82.0 | |
| | OLMo2-32B | 67.7 | 76.2 | 66.7 | 78.6 | 42.9 | 60.1 | 82.1 | |
| | EuroLLM-9B | 62.8 | 67.9 | 57.9 | 68.8 | 41.5 | 61.1 | 79.6 | |
| |
| Many additional benchmark evaluations (pretraining/post-training phases, |
| multilingual in ~100 languages, long-context) are in Section 5 of the |
| [Apertus Tech Report](https://arxiv.org/abs/2509.14233). |
| |
| **Source:** [official Apertus-70B-Instruct-2509 model card](https://huggingface.co/swiss-ai/Apertus-70B-Instruct-2509). |
| |
| > **Reading these alongside this LoRA:** Apertus-70B is EU AI Act-compliant |
| > (`Apertus_EU_Code_of_Practice.pdf`, `Apertus_EU_Public_Summary.pdf` included |
| > in upstream weights). This LoRA inherits that compliance plus the |
| > general-capability floor shown above, then adds domain specialization. |
|
|
