clemsail commited on
Commit
55dfd8e
·
verified ·
1 Parent(s): 169a508

docs: add Benchmark / Training metrics section

Browse files
Files changed (1) hide show
  1. README.md +18 -0
README.md CHANGED
@@ -66,6 +66,24 @@ print(generate(model, tokenizer, prompt="..."))
66
 
67
  For per-sample provenance and attribution status, consult the dataset card.
68
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
69
  ## License chain
70
 
71
  | Component | License |
 
66
 
67
  For per-sample provenance and attribution status, consult the dataset card.
68
 
69
+ ## Benchmark roadmap
70
+
71
+ This LoRA has **not yet been evaluated** through `electron-bench` (the current
72
+ pipeline supports `gemma-4-E4B` base only). Training was completed with the
73
+ standard `mlx-lm` LoRA trainer (rank 16, alpha 32, scale 2.0, AdamW
74
+ LR 1e-5, 500 iters) — full hyperparameters are in the `Training` table above.
75
+
76
+ Planned evaluations:
77
+
78
+ - Perplexity on the validation split of the training data
79
+ - Functional benchmark on **devstral**-specific tasks
80
+ - Comparison vs base `mistralai/Devstral-Small-2-24B-Instruct-2512`
81
+
82
+ Track progress: [ailiance-bench issues](https://github.com/ailiance/ailiance-bench/issues).
83
+
84
+ For reference benchmarks on the `gemma-4-E4B` base, see the
85
+ [base-vs-LoRA matrix](https://github.com/ailiance/ailiance-bench/blob/main/bench-results/compare_base_vs_lora.md).
86
+
87
  ## License chain
88
 
89
  | Component | License |