Update README.md

README.md

@@ -1,3 +1,60 @@
 ---
+language:
+- kab
+tags:
+- gpt2
+- causal-lm
+- custom-vocab
 license: mit
+datasets:
+- custom-kabyle-corpus
+metrics:
+- perplexity
 ---
+
+# Kabyle GPT-2 Base Model (Optimized BPE)
+
+This is a custom, lightweight GPT-2 style causal language model built from scratch specifically for the **Kabyle (Taqbaylit)** language. It utilizes a highly optimized morphological subword tokenizer trained with byte-aware rules to natively preserve and parse Latin-Tamazight text structures without visual noise artifacts.
+
+## Model Highlights
+* **Architecture:** Custom 8-layer, 8-attention-head Transformer (512 hidden dimensions) built from scratch.
+* **Context Window:** 256 tokens.
+* **Vocabulary Size:** 50,257 tokens.
+* **Tokenizer Efficiency:** Achieves an exceptional **97.95% vocabulary utilization rate** on native Kabyle corpuses, maximizing embedding row saturation and eliminating dead parameters common in massive multilingual tokenizers.
+
+## Tokenizer Performance
+
+Our custom Byte-Pair Encoding (BPE) pipeline maps linguistic affixes accurately. Compared to standard tokenizers that introduce raw byte visual noise (e.g., `ÉĽ`, `áºĵ`), this model correctly keeps character boundaries intact during inference:
+
+| Input Text Fragment | Standard Decoders (Noisy) | Our Native Pipeline (Clean) |
+| :--- | :--- | :--- |
+| **... yettɛawad ...** | `['yett', 'ÉĽawad']` | `['yett', 'ɛawad']` |
+| **... iẓerfan ...** | `['Ġiáºĵer', 'fan']` | `['Ġiẓer', 'fan']` |
+
+## Quickstart Usage
+
+You can load this model and its accompanying optimized tokenizer directly into your PyTorch environment:
+
+```python
+from transformers import PreTrainedTokenizerFast, GPT2LMHeadModel
+
+# Load the custom assets
+tokenizer = PreTrainedTokenizerFast.from_pretrained("boffire/kabyle-gpt2-tokenizer")
+model = GPT2LMHeadModel.from_pretrained("your-username/kabyle-llm-base")
+
+# Quick inference test
+text = "Wa d amcic-is aberkan,"
+inputs = tokenizer(text, return_tensors="pt")
+outputs = model.generate(**inputs, max_length=40, do_sample=True, top_k=50)
+
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+
+## Training Data & Methodology
+
+The model was pre-trained using a meticulously cleaned and normalized **Kabyle text corpus** (~20 MB / 5.01M total tokens). 
+
+### Optimization Settings
+* **Training Duration:** 3 Epochs
+* **Optimizer:** AdamW
+* **Learning Rate:** `5e-4`
+* **Batch Strategy:** Dynamic batch padding to maximize hardware and VRAM efficiency.