Update README: fix model references to HuggingFaceTB/nanowhale-*

README.md

@@ -1,6 +1,8 @@
-# SmolDeepSeek-V4 100M (Pretrained)
+# nanowhale-100m-base 🐳
 
-A small ~110M parameter language model implementing the **DeepSeek-V4 architecture** from scratch. This is the pretrained base model — see [cmpatino/smol-deepseek-v4-100m](https://huggingface.co/cmpatino/smol-deepseek-v4-100m) for the SFT/chat version.
+A small ~110M parameter language model implementing the **DeepSeek-V4 architecture** from scratch. This is the pretrained base model — see [HuggingFaceTB/nanowhale-100m](https://huggingface.co/HuggingFaceTB/nanowhale-100m) for the SFT/chat version.
+
+**Training code**: [github.com/huggingface/nanowhale](https://github.com/huggingface/nanowhale)
 
 ## Architecture
 
@@ -54,23 +56,27 @@ This model implements key DeepSeek-V4 innovations at a miniature scale:
 
 ```python
 import torch
-from transformers import AutoModelForCausalLM, AutoTokenizer
-
-model = AutoModelForCausalLM.from_pretrained(
-    "cmpatino/smol-deepseek-v4-100m-pretrain",
-    trust_remote_code=True,
-    torch_dtype=torch.float32,
-)
-tokenizer = AutoTokenizer.from_pretrained("cmpatino/smol-deepseek-v4-100m-pretrain")
-
-# Important: Use manual weight loading for best results
 from safetensors.torch import load_file
-from transformers import AutoConfig
+from transformers import AutoConfig, AutoModelForCausalLM, AutoTokenizer
+from huggingface_hub import hf_hub_download
 
-config = AutoConfig.from_pretrained("cmpatino/smol-deepseek-v4-100m-pretrain", trust_remote_code=True)
-model = AutoModelForCausalLM.from_config(config, trust_remote_code=True)
-state_dict = load_file("model.safetensors")  # or download from Hub
+# Load model (recommended: manual load for reliability)
+config = AutoConfig.from_pretrained("HuggingFaceTB/nanowhale-100m-base", trust_remote_code=True)
+model = AutoModelForCausalLM.from_config(config, trust_remote_code=True).float()
+
+# Download and load weights
+weights_path = hf_hub_download("HuggingFaceTB/nanowhale-100m-base", "model.safetensors")
+state_dict = load_file(weights_path)
 model.load_state_dict(state_dict, strict=True)
+model = model.cuda().eval()
+
+tokenizer = AutoTokenizer.from_pretrained("HuggingFaceTB/nanowhale-100m-base")
+
+# Generate
+input_ids = tokenizer.encode("The meaning of life is", return_tensors="pt").cuda()
+output = model.generate(input_ids, max_new_tokens=100, temperature=0.7, top_p=0.9,
+                        pad_token_id=tokenizer.eos_token_id)
+print(tokenizer.decode(output[0], skip_special_tokens=True))
 ```
 
 ## Limitations
@@ -78,6 +84,7 @@ model.load_state_dict(state_dict, strict=True)
 - **Small model**: 110M params with 129K vocab means ~37% of parameters are in embeddings, limiting model capacity
 - **Limited training**: Only 5K steps / 2.6B tokens — significantly undertrained compared to production models
 - **Pretrained only**: This is a base model without instruction tuning. Outputs are language-model completions, not conversations.
+- **bf16 NaN**: Use fp32 — the Hyper-Connections architecture produces values that overflow bf16 range at this scale.
 - **Custom architecture**: Requires `trust_remote_code=True`
 
 ## License