tue-mps/videomt-dinov2-large-vspw

VidEoMT-L on VSPW

This repository contains the Hugging Face Transformers conversion of the official VidEoMT checkpoint vspw_vit_large_95.0_64.9.pth from tue-mps/VidEoMT.

Model details

• Architecture: VidEoMT with a DINOv2 ViT-L/14 with 4 register tokens backbone
• Task: video semantic segmentation
• Dataset: VSPW
• Input resolution: 1280 x 1280
• Number of frames: 2
• Paper: Your ViT is Secretly Also a Video Segmentation Model

Reported metrics

Metric	Value
mVC_16	95.0
mIoU	64.9
FPS	73

The metrics above are the numbers reported by the authors in the official model zoo.

Usage

from transformers import AutoModelForUniversalSegmentation, AutoVideoProcessor

model_id = "tue-mps/videomt-dinov2-large-vspw"
processor = AutoVideoProcessor.from_pretrained(model_id)
model = AutoModelForUniversalSegmentation.from_pretrained(model_id)

Use processor.post_process_instance_segmentation, processor.post_process_panoptic_segmentation, or processor.post_process_semantic_segmentation depending on the target task.