clemsail commited on
Commit
3455f87
·
verified ·
1 Parent(s): 5b99719

docs: add Benchmark / Training metrics section

Browse files
Files changed (1) hide show
  1. README.md +18 -0
README.md CHANGED
@@ -54,6 +54,24 @@ Derived from the internal **eu-kiki / mascarade** curation. All upstream samples
54
  are synthetic, permissively-licensed, or generated from Apache-2.0 base resources.
55
  See the [Ailiance-fr catalog](https://huggingface.co/Ailiance-fr) for related cards.
56
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
57
  ## License chain
58
 
59
  | Component | License |
 
54
  are synthetic, permissively-licensed, or generated from Apache-2.0 base resources.
55
  See the [Ailiance-fr catalog](https://huggingface.co/Ailiance-fr) for related cards.
56
 
57
+ ## Benchmark roadmap
58
+
59
+ This LoRA has **not yet been evaluated** through `electron-bench` (the current
60
+ pipeline supports `gemma-4-E4B` base only). Training was completed with the
61
+ standard `mlx-lm` LoRA trainer (rank 16, alpha 32, scale 2.0, AdamW
62
+ LR 1e-5, 500 iters) — full hyperparameters are in the `Training` table above.
63
+
64
+ Planned evaluations:
65
+
66
+ - Perplexity on the validation split of the training data
67
+ - Functional benchmark on **devstral**-specific tasks
68
+ - Comparison vs base `mistralai/Devstral-Small-2-24B-Instruct-2512`
69
+
70
+ Track progress: [ailiance-bench issues](https://github.com/ailiance/ailiance-bench/issues).
71
+
72
+ For reference benchmarks on the `gemma-4-E4B` base, see the
73
+ [base-vs-LoRA matrix](https://github.com/ailiance/ailiance-bench/blob/main/bench-results/compare_base_vs_lora.md).
74
+
75
  ## License chain
76
 
77
  | Component | License |