PAWN-BASE-DISCARD-PLY

A causal transformer trained on random chess games, designed as a testbed for finetuning and augmentation methods at small scales.

Trained with discard_ply_limit=True (only naturally-ending games).

Model Details

Parameters	35.8M
Architecture	Decoder-only transformer (RMSNorm, SwiGLU, RoPE)
d_model	512
Layers	8
Heads	8
Vocabulary	4,278 tokens (4,096 grid + 176 promotions + 5 outcomes + 1 PAD)
Sequence length	256
Training steps	56.5K/100K
Best val loss	3.183 (step 56,535)
Best val accuracy	7.5%

Usage

import torch
from pawn.config import CLMConfig
from pawn.model import PAWNCLM

cfg = CLMConfig.base()
model = PAWNCLM(cfg)

ckpt = torch.load("model.pt", map_location="cpu", weights_only=False)
model.load_state_dict(ckpt["model_state_dict"])
model.eval()

Training

Trained from scratch on random self-play games generated by a Rust chess engine (shakmaty). See the PAWN repository for training code, data pipeline, and evaluation suite.

License

Apache 2.0