File size: 3,004 Bytes
f0d2e46 b3d54fb fc34167 f0d2e46 b3d54fb f0d2e46 b3d54fb f0d2e46 b3d54fb f0d2e46 b3d54fb f0d2e46 b3d54fb f0d2e46 b3d54fb f0d2e46 b3d54fb f0d2e46 b3d54fb f0d2e46 b3d54fb f0d2e46 b3d54fb f0d2e46 b3d54fb f0d2e46 b3d54fb f0d2e46 b3d54fb f0d2e46 b3d54fb f0d2e46 b3d54fb f0d2e46 b3d54fb f0d2e46 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 | ---
license: gpl-3.0
new_version: CompactAI-O/Glint-1
---
Note: You need the custom python script to run this model. Grab it from [the model runner Space](https://huggingface.co/spaces/CompactAI-O/CompactAIModelRunner).
# Glint-0.4
**It speaks. It actually speaks. Mostly.**
We came so far. From the dark ages of `couldcouldoldbloodblood` to actual, coherent sentences. This is Glint-0.4. 1 million parameters. Small. Trying its best. Unlike its ancestors, it usually succeeds.
## Quick Stats
- **Parameters:** 1,000,000 (yes, really)
- **Training Tokens:** 10 Billion
- **Context Window:** 2048 tokens
- **Vibe:** Chaotic good, but mostly good
## What Is This?
Glint-0.4 is the latest of the old guard before we started the Glint line proper. It builds on Glint-0.3 by adding SPIN (Self-Play Fine-Tuning) to the training loop. This model represents a 3x improvement in combined score over the original Glint-0.1. Coherence jumped from 1.99 to 6.03. Relevance is no longer zero. It is a miracle.
### The journey
| Model | Era | Typical Output | Combined |
| :--- | :--- | :--- | :--- |
| **Glint-0.1** | The Dark Ages | `couldcouldoldbloodbloodbodybody` | 1.62 |
| **Glint-0.2** | Pipe Character Incident | `\|fdish\|\|\|\|\|!@\|` | 1.21 |
| **Glint-0.3** | The Awakening | `It is about **competent development**...` | 3.87 |
| **Glint-0.4 (SPIN)** | Current Era | `The artificial intelligence is a problem...` | **4.84** |
**Expected output:**
> "The simple terms arrived in simulant explorers and honey are specific or forecasters. They allow the structure of their similar..."
## Disclaimer
This is a 1 million parameter model.
- It is not GPT-5.
- It is not GPT-2.
- It is a tiny neural network running on a prayer and a GPU.
- It might still output `chuamliamce`. If it does, try again. It is shy.
- For best results, use temperature around 0.7. At 2.0, you are on your own.
## Benchmarks
We benchmarked Glint-0.4 against all previous versions using a standard 7-question suite. Yes, 7 questions. We kept it small because we were running on a laptop.
| Metric | Glint-0.1 | Glint-0.2 | Glint-0.3 | **Glint-0.4 (SPIN)** |
| :--- | :---: | :---: | :---: | :---: |
| **Fluency** | 0.50 | 1.69 | 8.35 | **8.78** |
| **Coherence** | 1.99 | 1.56 | 5.72 | **6.03** |
| **Relevance** | 1.22 | 0.00 | 0.00 | **2.25** |
| **Format** | 3.29 | 3.29 | 3.29 | **3.29** |
| **Combined** | 1.62 | 1.21 | 3.87 | **4.84** |
## Related Models
- [Glint-0.1](https://huggingface.co/CompactAI-O/Glint-0.1) (the ancestor)
- [Glint-0.2](https://huggingface.co/CompactAI-O/Glint-0.2) (the pipe character one)
- [Glint-0.3](https://huggingface.co/CompactAI-O/Glint-0.3) (the breakthrough)
- [Glint-1](https://huggingface.co/CompactAI-O/Glint-1) (the current gen)
## Acknowledgments
Built with curiosity over compute. Trained on FineWeb-Edu. SPIN optimized. A lot of hope.
---
**Built by [CompactAI](https://huggingface.co/CompactAI-O).**
*If you like tiny models that try their best, give us a follow.*
|