OLMo-1B-as_fm3_tg_omi1_omi2_base

1B OLMo model pretrained on Algebraic Stack, FineMath3+, TinyGSM, OpenMathInstruct1, and OpenMathInstruct2.

Reading Model Names

Model names contain the contents of the pretraining dataset, delimited by underscores.

as: Algebraic Stack
fm3: FineMath3+
tg: TinyGSM
omi1: OpenMathInstruct1
omi2: OpenMathInstruct2

If there is a {n}x in front of the dataset abbreviation, that means it was repeated n times during pretraining. For instance, 2xtg refers to two passes over the TinyGSM dataset.

Downloads last month: 5

Safetensors

Model size

1B params

Tensor type

F32

Collections including rosieyzh/OLMo-1B-as_fm3_tg_omi1_omi2_base

OLMo-1B-as_fm3_tg_omi1_omi2

Collection

OLMo 1B model pretrained with Algebraic Stack, FineMath3, TinyGSM, OMI1, and OMI2. Includes checkpoints from doing PPO using GSM8K train. • 25 items • Updated Jan 26 • 2

OLMo-150M and OLMo-1B Pretrained Models

Collection

Pretrained models from scratch used in "Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretraining". • 12 items • Updated Jan 26 • 4