---
license: mit
tags:
  - kornia
  - image-classification
  - backbone
---

# kornia/tiny_vit

Pretrained weights for **TinyViT**,
used as the encoder backbone in
[`kornia.models.SegmentAnything`](https://kornia.readthedocs.io/en/latest/models.html)
(MobileSAM) and available via
[`kornia.models.TinyViT`](https://kornia.readthedocs.io/en/latest/models.html).

TinyViT is a small Vision Transformer trained with knowledge distillation from large
teacher models on ImageNet-22K. ECCV 2022.

**Original repo:** [microsoft/Cream/TinyViT](https://github.com/microsoft/Cream/tree/main/TinyViT)

## Weights

| File | Params | Pre-training | Fine-tuning |
|------|--------|-------------|-------------|
| `tiny_vit_5m_22k_distill.pth` | 5M | ImageNet-22K | — |
| `tiny_vit_5m_22kto1k_distill.pth` | 5M | ImageNet-22K | ImageNet-1K 224 |
| `tiny_vit_11m_22k_distill.pth` | 11M | ImageNet-22K | — |
| `tiny_vit_11m_22kto1k_distill.pth` | 11M | ImageNet-22K | ImageNet-1K 224 |
| `tiny_vit_21m_22k_distill.pth` | 21M | ImageNet-22K | — |
| `tiny_vit_21m_22kto1k_distill.pth` | 21M | ImageNet-22K | ImageNet-1K 224 |
| `tiny_vit_21m_22kto1k_384_distill.pth` | 21M | ImageNet-22K | ImageNet-1K 384 |
| `tiny_vit_21m_22kto1k_512_distill.pth` | 21M | ImageNet-22K | ImageNet-1K 512 |

## Citation

```bibtex
@inproceedings{wu2022tinyvit,
    title     = {{TinyViT}: Fast Pretraining Distillation for Small Vision Transformers},
    author    = {Wu, Kan and Zhang, Jinnian and Peng, Houwen and Liu, Mengchen
                 and Xiao, Bin and Fu, Jianlong and Yuan, Lu},
    booktitle = {ECCV},
    year      = {2022}
}
```