Fixes: citation, grant, training time, eval table bolds, terminology cleanup

README.md

@@ -189,14 +189,14 @@ outputs = model(**inputs)
 
 ModernBERT-bio-large is trained in two phases, initialized from [ModernBERT-large](https://huggingface.co/answerdotai/ModernBERT-large):
 
-* **Phase 1 — CLM detour (50B tokens):** The bidirectional attention mask is replaced with a causal mask, and the model is trained with next-token prediction. This dense training signal (100% of positions) deeply modifies early transformer layers for domain adaptation.
-* **Phase 2 — MLM decay (5B tokens):** Bidirectional attention is restored, and the model is trained with masked language modeling at 15% masking. The learning rate decays from peak to 10% following a 1-sqrt schedule.
+* **Phase 1 (CLM detour, 50B tokens):** The bidirectional attention mask is replaced with a causal mask, and the model is trained with next-token prediction. This dense training signal (100% of positions) deeply modifies early transformer layers for domain adaptation.
+* **Phase 2 (MLM decay, 5B tokens):** Bidirectional attention is restored, and the model is trained with masked language modeling at 15% masking. The learning rate decays from peak to 10% following a 1-sqrt schedule.
 
-Both phases use the same data mix (55B tokens total). Training used AdamW (lr=2e-4, beta1=0.9, beta2=0.98), bf16 mixed precision, global batch size of 384 sequences (~3.1M tokens), on 4× H100 80GB GPUs with [Composer](https://github.com/mosaicml/composer). Total training time: ~8 GPU-hours.
+Both phases use the same data mix (55B tokens total). Training used AdamW (lr=2e-4, beta1=0.9, beta2=0.98), bf16 mixed precision, global batch size of 384 sequences (~3.1M tokens), on 4× H100 80GB GPUs with [Composer](https://github.com/mosaicml/composer).
 
 ### Why a CLM Detour?
 
-CLM supervises every token position, producing dense gradient updates that deeply modify early transformer layers. These changes persist through the MLM decay phase — a phenomenon we call **computational hysteresis**. The Large model retains 67.2% CKA divergence from its MLM counterpart (vs 56.5% for Base), showing that hysteresis scales with model capacity. The CLM benefit also widens at Large scale: +0.8pp (Large) vs +0.3pp (Base). See our paper for the full mechanistic analysis.
+CLM supervises every token position, producing dense gradient updates that deeply modify early transformer layers. These changes persist through the MLM decay phase, even when the decay matches the CLM phase in length. The Large model retains 67.2% CKA divergence from its MLM counterpart (vs 56.5% for Base), showing that the effect scales with model capacity. The CLM benefit also widens at Large scale: +0.8pp (Large) vs +0.3pp (Base). See our paper for the full mechanistic analysis.
 
 ## Evaluation
 
@@ -206,7 +206,7 @@ English biomedical benchmark results (11 tasks, 5 seeds per model):
 
 | Model | Ctx | ChemProt | Phenotype | COS | Social Hist. | DEID | **Avg** |
 |-------|-----|----------|-----------|-----|-------------|------|---------|
-| **ModernBERT-bio-large** | 8192 | 90.4 | 61.3 | 94.7 | **56.5** | **84.2** | **77.4** |
+| **ModernBERT-bio-large** | 8192 | 90.4 | **61.3** | 94.7 | **56.5** | **84.2** | **77.4** |
 | MLM baseline Large (ours) | 8192 | **90.5** | 61.0 | 94.9 | 55.0 | 82.3 | 76.7 |
 | BioClinical-ModernBERT-base | 8192 | 90.0 | 60.7 | 94.8 | 56.0 | 81.8 | 76.7 |
 | PubMedBERT | 512 | 90.2 | 52.0 | **95.0** | 48.7 | 80.4 | 73.3 |
@@ -216,9 +216,9 @@ English biomedical benchmark results (11 tasks, 5 seeds per model):
 | Model | Ctx | AnatEM | BC5CDR | JNLPBA | NCBI | GAD | HoC | **Avg** |
 |-------|-----|--------|--------|--------|------|-----|-----|---------|
 | **ModernBERT-bio-large** | 8192 | **83.2** | **89.8** | 75.3 | 81.7 | **79.7** | 69.3 | **79.8** |
-| MLM baseline Large (ours) | 8192 | 82.0 | 89.4 | **75.5** | **81.8** | 76.4 | 67.8 | 78.8 |
+| MLM baseline Large (ours) | 8192 | 82.0 | 89.4 | **75.5** | 81.8 | 76.4 | 67.8 | 78.8 |
 | BioClinical-ModernBERT-base | 8192 | 79.2 | 88.7 | 74.8 | 78.7 | 75.8 | 67.0 | 77.4 |
-| PubMedBERT | 512 | 83.3 | 89.7 | 74.9 | 82.1 | 79.3 | **71.0** | 80.1 |
+| PubMedBERT | 512 | 83.3 | 89.7 | 74.9 | **82.1** | 79.3 | **71.0** | 80.1 |
 
 ### Overall
 
@@ -230,7 +230,7 @@ English biomedical benchmark results (11 tasks, 5 seeds per model):
 | BioClinical-ModernBERT-base | 76.7 | 77.4 | 77.0 |
 | PubMedBERT | 73.3 | 80.1 | 77.0 |
 
-ModernBERT-bio-large achieves the highest overall score (78.7%), with the CLM benefit widening at Large scale (+0.8pp vs +0.3pp for Base). The model sets new state-of-the-art on DEID (84.2%), BC5CDR (89.8%), GAD (79.7%), and Social History (56.5%).
+ModernBERT-bio-large achieves the highest overall score (78.7%), with the CLM benefit widening at Large scale (+0.8pp vs +0.3pp for Base). The model sets new state-of-the-art on DEID (84.2%) and is competitive with the best baselines on the remaining tasks.
 
 ## Intended Use
 
@@ -265,14 +265,14 @@ Apache 2.0
 ## Citation
 
 ```bibtex
-@inproceedings{touchent2026clm,
+@article{touchent2026clmdetour,
   title={A Causal Language Modeling Detour Improves Encoder Continued Pretraining},
   author={Touchent, Rian and de la Clergerie, {\'E}ric},
-  booktitle={Proceedings of COLM},
-  year={2026}
+  year={2026},
+  journal={arXiv preprint}
 }
 ```
 
 ## Acknowledgments
 
-This work was performed using HPC resources from GENCI-IDRIS (Grant 2024-AD011015883).
+This work was performed using HPC resources from GENCI-IDRIS (Grant 2024-AD011014393R2).