| --- |
| base_model: h2oai/h2o-danube3-500m-base |
| language: |
| - en |
| license: apache-2.0 |
| tags: |
| - text-generation-inference |
| - transformers |
| - unsloth |
| - llama |
| - trl |
| - sft |
| --- |
| |
| # Uploaded model |
|
|
| base model pretrained on 2.5 epochs on no-prompt-oasst |
|
|
| hf (pretrained=appvoid/palmer-004-turbo-v1.2), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: 8 |
| | Tasks |Version|Filter|n-shot| Metric | |Value | |Stderr| |
| |---------------------------------------|------:|------|-----:|--------|---|-----:|---|-----:| |
| |arc_challenge | 1|none | 0|acc |↑ |0.3097|± |0.0135| |
| | | |none | 0|acc_norm|↑ |0.3464|± |0.0139| |
| |hellaswag | 1|none | 0|acc |↑ |0.4660|± |0.0050| |
| | | |none | 0|acc_norm|↑ |0.6130|± |0.0049| |
| |mmlu | 1|none | |acc |↑ |0.2728|± |0.0037| |
| | - humanities | 1|none | |acc |↑ |0.2593|± |0.0064| |
| | - formal_logic | 0|none | 0|acc |↑ |0.3175|± |0.0416| |
| | - high_school_european_history | 0|none | 0|acc |↑ |0.2970|± |0.0357| |
| | - high_school_us_history | 0|none | 0|acc |↑ |0.2941|± |0.0320| |
| | - high_school_world_history | 0|none | 0|acc |↑ |0.2489|± |0.0281| |
| | - international_law | 0|none | 0|acc |↑ |0.3306|± |0.0429| |
| | - jurisprudence | 0|none | 0|acc |↑ |0.2685|± |0.0428| |
| | - logical_fallacies | 0|none | 0|acc |↑ |0.2515|± |0.0341| |
| | - moral_disputes | 0|none | 0|acc |↑ |0.2486|± |0.0233| |
| | - moral_scenarios | 0|none | 0|acc |↑ |0.2380|± |0.0142| |
| | - philosophy | 0|none | 0|acc |↑ |0.2797|± |0.0255| |
| | - prehistory | 0|none | 0|acc |↑ |0.2593|± |0.0244| |
| | - professional_law | 0|none | 0|acc |↑ |0.2555|± |0.0111| |
| | - world_religions | 0|none | 0|acc |↑ |0.2339|± |0.0325| |
| | - other | 1|none | |acc |↑ |0.2813|± |0.0080| |
| | - business_ethics | 0|none | 0|acc |↑ |0.1700|± |0.0378| |
| | - clinical_knowledge | 0|none | 0|acc |↑ |0.3434|± |0.0292| |
| | - college_medicine | 0|none | 0|acc |↑ |0.3006|± |0.0350| |
| | - global_facts | 0|none | 0|acc |↑ |0.3600|± |0.0482| |
| | - human_aging | 0|none | 0|acc |↑ |0.1659|± |0.0250| |
| | - management | 0|none | 0|acc |↑ |0.3883|± |0.0483| |
| | - marketing | 0|none | 0|acc |↑ |0.2137|± |0.0269| |
| | - medical_genetics | 0|none | 0|acc |↑ |0.2500|± |0.0435| |
| | - miscellaneous | 0|none | 0|acc |↑ |0.2784|± |0.0160| |
| | - nutrition | 0|none | 0|acc |↑ |0.2386|± |0.0244| |
| | - professional_accounting | 0|none | 0|acc |↑ |0.2376|± |0.0254| |
| | - professional_medicine | 0|none | 0|acc |↑ |0.4265|± |0.0300| |
| | - virology | 0|none | 0|acc |↑ |0.3133|± |0.0361| |
| | - social sciences | 1|none | |acc |↑ |0.2847|± |0.0081| |
| | - econometrics | 0|none | 0|acc |↑ |0.2719|± |0.0419| |
| | - high_school_geography | 0|none | 0|acc |↑ |0.3788|± |0.0346| |
| | - high_school_government_and_politics| 0|none | 0|acc |↑ |0.2850|± |0.0326| |
| | - high_school_macroeconomics | 0|none | 0|acc |↑ |0.3359|± |0.0239| |
| | - high_school_microeconomics | 0|none | 0|acc |↑ |0.3361|± |0.0307| |
| | - high_school_psychology | 0|none | 0|acc |↑ |0.3064|± |0.0198| |
| | - human_sexuality | 0|none | 0|acc |↑ |0.2519|± |0.0381| |
| | - professional_psychology | 0|none | 0|acc |↑ |0.2190|± |0.0167| |
| | - public_relations | 0|none | 0|acc |↑ |0.3091|± |0.0443| |
| | - security_studies | 0|none | 0|acc |↑ |0.2449|± |0.0275| |
| | - sociology | 0|none | 0|acc |↑ |0.2388|± |0.0301| |
| | - us_foreign_policy | 0|none | 0|acc |↑ |0.2800|± |0.0451| |
| | - stem | 1|none | |acc |↑ |0.2731|± |0.0079| |
| | - abstract_algebra | 0|none | 0|acc |↑ |0.1900|± |0.0394| |
| | - anatomy | 0|none | 0|acc |↑ |0.1926|± |0.0341| |
| | - astronomy | 0|none | 0|acc |↑ |0.2829|± |0.0367| |
| | - college_biology | 0|none | 0|acc |↑ |0.2083|± |0.0340| |
| | - college_chemistry | 0|none | 0|acc |↑ |0.3400|± |0.0476| |
| | - college_computer_science | 0|none | 0|acc |↑ |0.3000|± |0.0461| |
| | - college_mathematics | 0|none | 0|acc |↑ |0.3100|± |0.0465| |
| | - college_physics | 0|none | 0|acc |↑ |0.2941|± |0.0453| |
| | - computer_security | 0|none | 0|acc |↑ |0.2400|± |0.0429| |
| | - conceptual_physics | 0|none | 0|acc |↑ |0.2553|± |0.0285| |
| | - electrical_engineering | 0|none | 0|acc |↑ |0.2828|± |0.0375| |
| | - elementary_mathematics | 0|none | 0|acc |↑ |0.2513|± |0.0223| |
| | - high_school_biology | 0|none | 0|acc |↑ |0.2935|± |0.0259| |
| | - high_school_chemistry | 0|none | 0|acc |↑ |0.2808|± |0.0316| |
| | - high_school_computer_science | 0|none | 0|acc |↑ |0.2400|± |0.0429| |
| | - high_school_mathematics | 0|none | 0|acc |↑ |0.2741|± |0.0272| |
| | - high_school_physics | 0|none | 0|acc |↑ |0.3775|± |0.0396| |
| | - high_school_statistics | 0|none | 0|acc |↑ |0.3194|± |0.0318| |
| | - machine_learning | 0|none | 0|acc |↑ |0.2321|± |0.0401| |
| |piqa | 1|none | 0|acc |↑ |0.7269|± |0.0104| |
| | | |none | 0|acc_norm|↑ |0.7242|± |0.0104| |
| |winogrande | 1|none | 0|acc |↑ |0.5848|± |0.0138| |
|
|
| | Groups |Version|Filter|n-shot|Metric| |Value | |Stderr| |
| |------------------|------:|------|------|------|---|-----:|---|-----:| |
| |mmlu | 1|none | |acc |↑ |0.2728|± |0.0037| |
| | - humanities | 1|none | |acc |↑ |0.2593|± |0.0064| |
| | - other | 1|none | |acc |↑ |0.2813|± |0.0080| |
| | - social sciences| 1|none | |acc |↑ |0.2847|± |0.0081| |
| | - stem | 1|none | |acc |↑ |0.2731|± |0.0079| |
|
|
