cashbook/SOLAR-Platypus-10.7B-v1-kjw

Data

• kyujinpy/Open-platypus-Commercial

Base Model

• upstage/SOLAR-10.7B-v1.0

Zero-shot 평가

hf (pretrained=cashbook/SOLAR-Platypus-10.7B-v1-kjw), gen_kwargs: (None), limit: None, num_fewshot: 0, batch_size: 8

Tasks	Version	Filter	n-shot	Metric	Value		Stderr
mmlu	N/A	none	0	acc	0.6304	±	0.0038
- humanities	N/A	none	0	acc	0.5626	±	0.0066
- formal_logic	0	none	0	acc	0.3413	±	0.0424
- high_school_european_history	0	none	0	acc	0.7818	±	0.0323
- high_school_us_history	0	none	0	acc	0.8284	±	0.0265
- high_school_world_history	0	none	0	acc	0.8101	±	0.0255
- international_law	0	none	0	acc	0.8099	±	0.0358
- jurisprudence	0	none	0	acc	0.7407	±	0.0424
- logical_fallacies	0	none	0	acc	0.7607	±	0.0335
- moral_disputes	0	none	0	acc	0.7312	±	0.0239
- moral_scenarios	0	none	0	acc	0.2413	±	0.0143
- philosophy	0	none	0	acc	0.7074	±	0.0258
- prehistory	0	none	0	acc	0.7500	±	0.0241
- professional_law	0	none	0	acc	0.4831	±	0.0128
- world_religions	0	none	0	acc	0.8129	±	0.0299
- other	N/A	none	0	acc	0.7219	±	0.0077
- business_ethics	0	none	0	acc	0.7000	±	0.0461
- clinical_knowledge	0	none	0	acc	0.6981	±	0.0283
- college_medicine	0	none	0	acc	0.6474	±	0.0364
- global_facts	0	none	0	acc	0.3600	±	0.0482
- human_aging	0	none	0	acc	0.7175	±	0.0302
- management	0	none	0	acc	0.7961	±	0.0399
- marketing	0	none	0	acc	0.8932	±	0.0202
- medical_genetics	0	none	0	acc	0.7800	±	0.0416
- miscellaneous	0	none	0	acc	0.8340	±	0.0133
- nutrition	0	none	0	acc	0.7516	±	0.0247
- professional_accounting	0	none	0	acc	0.5319	±	0.0298
- professional_medicine	0	none	0	acc	0.7022	±	0.0278
- virology	0	none	0	acc	0.5241	±	0.0389
- social_sciences	N/A	none	0	acc	0.7423	±	0.0077
- econometrics	0	none	0	acc	0.4737	±	0.0470
- high_school_geography	0	none	0	acc	0.8131	±	0.0278
- high_school_government_and_politics	0	none	0	acc	0.8756	±	0.0238
- high_school_macroeconomics	0	none	0	acc	0.6308	±	0.0245
- high_school_microeconomics	0	none	0	acc	0.7269	±	0.0289
- high_school_psychology	0	none	0	acc	0.8367	±	0.0158
- human_sexuality	0	none	0	acc	0.7786	±	0.0364
- professional_psychology	0	none	0	acc	0.6667	±	0.0191
- public_relations	0	none	0	acc	0.7000	±	0.0439
- security_studies	0	none	0	acc	0.7388	±	0.0281
- sociology	0	none	0	acc	0.8507	±	0.0252
- us_foreign_policy	0	none	0	acc	0.8600	±	0.0349
- stem	N/A	none	0	acc	0.5322	±	0.0086
- abstract_algebra	0	none	0	acc	0.3300	±	0.0473
- anatomy	0	none	0	acc	0.5926	±	0.0424
- astronomy	0	none	0	acc	0.7039	±	0.0372
- college_biology	0	none	0	acc	0.7708	±	0.0351
- college_chemistry	0	none	0	acc	0.4100	±	0.0494
- college_computer_science	0	none	0	acc	0.5400	±	0.0501
- college_mathematics	0	none	0	acc	0.3800	±	0.0488
- college_physics	0	none	0	acc	0.4118	±	0.0490
- computer_security	0	none	0	acc	0.7300	±	0.0446
- conceptual_physics	0	none	0	acc	0.5362	±	0.0326
- electrical_engineering	0	none	0	acc	0.5655	±	0.0413
- elementary_mathematics	0	none	0	acc	0.4339	±	0.0255
- high_school_biology	0	none	0	acc	0.7742	±	0.0238
- high_school_chemistry	0	none	0	acc	0.4975	±	0.0352
- high_school_computer_science	0	none	0	acc	0.6200	±	0.0488
- high_school_mathematics	0	none	0	acc	0.3593	±	0.0293
- high_school_physics	0	none	0	acc	0.3974	±	0.0400
- high_school_statistics	0	none	0	acc	0.5509	±	0.0339
- machine_learning	0	none	0	acc	0.4286	±	0.0470
hellaswag	1	none	0	acc	0.6396	±	0.0048
		none	0	acc_norm	0.8310	±	0.0037
copa	1	none	0	acc	0.8700	±	0.0338
boolq	2	none	0	acc	0.8260	±	0.0066

Groups	Version	Filter	n-shot	Metric	Value		Stderr
mmlu	N/A	none	0	acc	0.6304	±	0.0038
- humanities	N/A	none	0	acc	0.5626	±	0.0066
- other	N/A	none	0	acc	0.7219	±	0.0077
- social_sciences	N/A	none	0	acc	0.7423	±	0.0077
- stem	N/A	none	0	acc	0.5322	±	0.0086