Kiel-26M-Matured

⚠️ Status: research artifact / early experiment, not a usable language model today. The tokenizer used at training time was not preserved in this repository, so the checkpoint cannot be loaded end-to-end with HF AutoModel / AutoTokenizer. Output quality on standard Indonesian benchmarks is far below the org's working models (Kiel-Pro-0.5B-v3, AksaraLLM-Qwen-1.5B-v5-public).

What this is

A 39M-parameter decoder-only transformer trained from scratch as part of the early AksaraLLM experiments. Architecture (inferred from weight shapes):

Property	Value
Parameters	38.7M
Layers	6
Heads	6
Hidden size	384
FFN size (SwiGLU)	1536
Vocabulary	32000
Context length	256
RMSNorm + RoPE + SwiGLU	yes

Measured baseline (Devin audit, CPU eval)

We loaded this checkpoint with aksarallm.model.aksaraLLMModel, tested several candidate tokenizers (AksaraLLM/aksara-tokenizer-v1/v2/v3, Llama-2 SentencePiece, GPT-2 BPE), and ran perplexity on 50 short Indonesian Wikipedia-style sentences plus 5 free-form prompts. Best-case results:

• Perplexity: BROKEN (tokenizer mismatch — see Limitations) — meaning the model is not modelling Indonesian distribution.
• English-stopword ratio in ID-prompted output: 0.0%
• Indonesian-stopword ratio in ID-prompted output: 0.0%

Sample output for prompt "Indonesia adalah negara":

Indonesia adalah negara2 March enough pur streCon compens4!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Why the original "Skor 10/11 (Grade S)" claim is misleading

The score in earlier versions of this README is from a custom 11-question in-house scorecard graded on a tiny SFT set, not from a standard language modelling metric. By any standard LM evaluation (perplexity, response coherence on out-of-distribution prompts, identity calibration), this model does not function.

Limitations

• Tokenizer not preserved. Without it, downstream usage is impossible.
• No HF-compatible config.json in the original training pipeline; the aksarallm package is required for loading.
• Vocab size 32000 — does not match any published AksaraLLM tokenizer (32 768, 64 000) or common open tokenizers (Llama-2 = 32 000).
• Trained on a small mixed corpus (Wikipedia / SFT pairs), insufficient for general Indonesian generation at this scale.

What to use instead

If you want a small Indonesian LM in the AksaraLLM family, use:

• AksaraLLM/Kiel-Pro-0.5B-v3 — 494M Qwen2-based, perplexity ≈ 15 on the same eval set.
• AksaraLLM/AksaraLLM-Qwen-1.5B-v5-public — 1.78B Qwen2-based, perplexity ≈ 8.4.

Citation

@misc{aksarallm-kiel-26m-matured,
  author       = {Cahyok Putra and AksaraLLM Community},
  title        = {Kiel-26M-Matured: early-experiment Indonesian transformer (39M params)},
  year         = 2025,
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/AksaraLLM/Kiel-26M-Matured}},
}

License

Apache 2.0