aynetdia commited on
Commit
5dd8efe
·
verified ·
1 Parent(s): 5c175e7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -5,7 +5,7 @@ language:
5
  pipeline_tag: text-generation
6
  library_name: transformers
7
  ---
8
- # Boldt-1B
9
 
10
  <img src="logo.png" width="500">
11
 
@@ -43,7 +43,7 @@ outputs = model.generate(**inputs, max_new_tokens=64)
43
  ## Evaluation
44
 
45
  We evaluate Boldt-350M on our [modernized German benchmark suite](https://huggingface.co/collections/Boldt/german-llm-benchmarks). It comprises the German subset of [Global MMLU](https://huggingface.co/datasets/CohereLabs/Global-MMLU) and updated translations of widely used English benchmarks, produced using [Tower+ 72B](https://huggingface.co/Unbabel/Tower-Plus-72B) to address issues we identified in existing German benchmark translations.
46
- Despite being trained on a substantially smaller amount of data, Boldt-1B outperforms other similarly-sized SLMs capable of German on our evaluation suite. It also performs competitively with larger-sized (around 1B) German and multilingual models.
47
 
48
  | Category | Model | Tokens | MMLU | ARC-C | ARC-E | H-Swag | LAMBADA | OBQA | Avg. |
49
  |----------|--------|--------|------|-------|-------|--------|----------|------|------|
 
5
  pipeline_tag: text-generation
6
  library_name: transformers
7
  ---
8
+ # Boldt-350M
9
 
10
  <img src="logo.png" width="500">
11
 
 
43
  ## Evaluation
44
 
45
  We evaluate Boldt-350M on our [modernized German benchmark suite](https://huggingface.co/collections/Boldt/german-llm-benchmarks). It comprises the German subset of [Global MMLU](https://huggingface.co/datasets/CohereLabs/Global-MMLU) and updated translations of widely used English benchmarks, produced using [Tower+ 72B](https://huggingface.co/Unbabel/Tower-Plus-72B) to address issues we identified in existing German benchmark translations.
46
+ Despite being trained on a substantially smaller amount of data, Boldt-350M outperforms other similarly-sized SLMs capable of German on our evaluation suite. It also performs competitively with larger-sized (around 1B) German and multilingual models.
47
 
48
  | Category | Model | Tokens | MMLU | ARC-C | ARC-E | H-Swag | LAMBADA | OBQA | Avg. |
49
  |----------|--------|--------|------|-------|-------|--------|----------|------|------|