| --- |
| license: gpl-3.0 |
| new_version: CompactAI-O/Glint-1 |
| --- |
| |
| Note: You need the custom python script to run this model. Grab it from [the model runner Space](https://huggingface.co/spaces/CompactAI-O/CompactAIModelRunner). |
|
|
| # Glint-0.4 |
|
|
| **It speaks. It actually speaks. Mostly.** |
|
|
| We came so far. From the dark ages of `couldcouldoldbloodblood` to actual, coherent sentences. This is Glint-0.4. 1 million parameters. Small. Trying its best. Unlike its ancestors, it usually succeeds. |
|
|
| ## Quick Stats |
|
|
| - **Parameters:** 1,000,000 (yes, really) |
| - **Training Tokens:** 10 Billion |
| - **Context Window:** 2048 tokens |
| - **Vibe:** Chaotic good, but mostly good |
|
|
| ## What Is This? |
|
|
| Glint-0.4 is the latest of the old guard before we started the Glint line proper. It builds on Glint-0.3 by adding SPIN (Self-Play Fine-Tuning) to the training loop. This model represents a 3x improvement in combined score over the original Glint-0.1. Coherence jumped from 1.99 to 6.03. Relevance is no longer zero. It is a miracle. |
|
|
| ### The journey |
|
|
| | Model | Era | Typical Output | Combined | |
| | :--- | :--- | :--- | :--- | |
| | **Glint-0.1** | The Dark Ages | `couldcouldoldbloodbloodbodybody` | 1.62 | |
| | **Glint-0.2** | Pipe Character Incident | `\|fdish\|\|\|\|\|!@\|` | 1.21 | |
| | **Glint-0.3** | The Awakening | `It is about **competent development**...` | 3.87 | |
| | **Glint-0.4 (SPIN)** | Current Era | `The artificial intelligence is a problem...` | **4.84** | |
|
|
| **Expected output:** |
| > "The simple terms arrived in simulant explorers and honey are specific or forecasters. They allow the structure of their similar..." |
|
|
| ## Disclaimer |
|
|
| This is a 1 million parameter model. |
| - It is not GPT-5. |
| - It is not GPT-2. |
| - It is a tiny neural network running on a prayer and a GPU. |
| - It might still output `chuamliamce`. If it does, try again. It is shy. |
| - For best results, use temperature around 0.7. At 2.0, you are on your own. |
|
|
| ## Benchmarks |
|
|
| We benchmarked Glint-0.4 against all previous versions using a standard 7-question suite. Yes, 7 questions. We kept it small because we were running on a laptop. |
|
|
| | Metric | Glint-0.1 | Glint-0.2 | Glint-0.3 | **Glint-0.4 (SPIN)** | |
| | :--- | :---: | :---: | :---: | :---: | |
| | **Fluency** | 0.50 | 1.69 | 8.35 | **8.78** | |
| | **Coherence** | 1.99 | 1.56 | 5.72 | **6.03** | |
| | **Relevance** | 1.22 | 0.00 | 0.00 | **2.25** | |
| | **Format** | 3.29 | 3.29 | 3.29 | **3.29** | |
| | **Combined** | 1.62 | 1.21 | 3.87 | **4.84** | |
|
|
| ## Related Models |
|
|
| - [Glint-0.1](https://huggingface.co/CompactAI-O/Glint-0.1) (the ancestor) |
| - [Glint-0.2](https://huggingface.co/CompactAI-O/Glint-0.2) (the pipe character one) |
| - [Glint-0.3](https://huggingface.co/CompactAI-O/Glint-0.3) (the breakthrough) |
| - [Glint-1](https://huggingface.co/CompactAI-O/Glint-1) (the current gen) |
|
|
| ## Acknowledgments |
|
|
| Built with curiosity over compute. Trained on FineWeb-Edu. SPIN optimized. A lot of hope. |
|
|
| --- |
|
|
| **Built by [CompactAI](https://huggingface.co/CompactAI-O).** |
| *If you like tiny models that try their best, give us a follow.* |
|
|
