gyanai/ayn-88M-hf

Description

Ayn is an 88 million parameter generative, decoder-only tiny legal language model for the Indian Supreme Court.

This model was pretrained from scratch for 185 A100 GPU hours with a context size of 8192 tokens.

This model is not either chat-tuned or fine-tuned.

We recommend to fine-tune/chat-tune this pretrained model on legal instruction datasets. Please use PyTorch for fine-tuning/instruction-tuning.

This model is strictly prohibited from being used for commercial purposes.

If you use our model, please cite our paper Niyogi et al., 2026

Model Architecture

Transformer Decoder-Only Auto Regressive Model

Limitations

Our legal model is currently limited to the Supreme Court of India and has not been exposed to various jurisdictions, including district and High Court case documents in India. Moreover, the legal model and tokenizer have been trained only on English court documents, which restricts their effectiveness for multilingual legal documents and knowledge in Indian languages. The tokenizer can be extended to support multilinguality, but with additional training. We also lack legal expert human evaluation for the legal case summarization task. As a result, the model may generate biased opinions, factually incorrect information, or hallucinations due to its generative nature. Additionally, we did not anonymize publicly available legal cases during tokenization and pretraining.

Citations

@misc{niyogi2026ayntinycompetitiveindian,
      title={Ayn: A Tiny yet Competitive Indian Legal Language Model Pretrained from Scratch}, 
      author={Mitodru Niyogi and Eric Gaussier and Arnab Bhattacharya},
      year={2026},
      eprint={2403.13681},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2403.13681}, 
}