gyanai/paramanu-oriya-87M-hf

Description

Paramanu-Oriya is an 87 million-parameter, open-source, monolingual Oriya decoder-only autoregressive language model.

It is pretrained from scratch on an open-source Oriya corpus with a context size of 1024 tokens.

It is neither chat-tuned nor fine-tuned; we recommend fine-tuning or chat-tuning Oriya datasets using PyTorch.

Commercial use is prohibited.

If you use our model, please cite our paper: Niyogi et al., 2026.

Model Architecture

Transformer Decoder Auto Regressive Model

Limitations

The model was trained on data containing toxic language, unsafe content, and societal biases originally crawled from the internet. Therefore, it may amplify these biases and produce toxic responses, especially when prompted with toxic inputs. The model may also generate answers that are inaccurate, omit key information, or include irrelevant or redundant text, potentially producing socially unacceptable or undesirable content, even when the prompt itself is not explicitly offensive.

Citations

@misc{niyogi2026paramanucompactcompetitivemonolingual,
      title={Paramanu: Compact and Competitive Monolingual Language Models for Low-Resource Morphologically Rich Indian Languages}, 
      author={Mitodru Niyogi and Eric Gaussier and Arnab Bhattacharya},
      year={2026},
      eprint={2401.18034},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2401.18034}, 
}