Upload EVAL50_README.md with huggingface_hub
Browse files- EVAL50_README.md +11 -0
EVAL50_README.md
ADDED
|
@@ -0,0 +1,11 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Gemma-4-E4B-it-Coder 50-Problem Eval
|
| 2 |
+
|
| 3 |
+
Executable HumanEval subset: 50 tasks.
|
| 4 |
+
|
| 5 |
+
| Metric | Base | Coder |
|
| 6 |
+
|---|---:|---:|
|
| 7 |
+
| Pass count | 34 / 50 | 42 / 50 |
|
| 8 |
+
| Absolute lift | - | 16.0 pp |
|
| 9 |
+
| Relative pass-count lift | - | 23.53% |
|
| 10 |
+
|
| 11 |
+
Full generated code is in `eval50_before_after_full_code.csv`.
|