Add pipeline tag, links and usage instructions

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +38 -5
README.md CHANGED
@@ -1,15 +1,48 @@
1
  ---
2
  license: apache-2.0
 
3
  ---
4
 
 
5
 
6
- ## OmniShotCut: Holistic Relational Shot Boundary Detection with Shot-Query Transformer
7
-
8
- OmniShotCut is a sensitive and more informative SoTA on the Shot Boundary Detection. \
9
- OmniShotCut can detect shot changes of the video in diverse sources (anime, vlog, game, shorts, sports, screen recording, etc.), and recognize Sudden Jump and Transitions (dissolve, fade, wipe, etc.) by proposing a Shot-Query-based Video Transformer.
10
-
11
 
12
  [![Paper](https://img.shields.io/badge/arXiv-Paper-b31b1b?logo=arxiv&logoColor=white)](https://arxiv.org/abs/2604.24762)
13
  [![Website](https://img.shields.io/badge/Project-Website-pink?logo=googlechrome&logoColor=white)](https://uva-computer-vision-lab.github.io/OmniShotCut_website/)
 
14
  <a href="https://huggingface.co/spaces/uva-cv-lab/OmniShotCut"><img src="https://img.shields.io/static/v1?label=%F0%9F%A4%97%20HF%20Space&message=Online+Demo&color=orange"></a>
15
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ pipeline_tag: video-classification
4
  ---
5
 
6
+ # OmniShotCut: Holistic Relational Shot Boundary Detection with Shot-Query Transformer
7
 
8
+ OmniShotCut is a sensitive and informative state-of-the-art (SoTA) model for Shot Boundary Detection (SBD). It can detect shot changes in videos from diverse sources (anime, vlog, game, shorts, sports, screen recording, etc.) and recognize Sudden Jumps and Transitions (dissolve, fade, wipe, etc.) by proposing a Shot-Query-based Video Transformer.
 
 
 
 
9
 
10
  [![Paper](https://img.shields.io/badge/arXiv-Paper-b31b1b?logo=arxiv&logoColor=white)](https://arxiv.org/abs/2604.24762)
11
  [![Website](https://img.shields.io/badge/Project-Website-pink?logo=googlechrome&logoColor=white)](https://uva-computer-vision-lab.github.io/OmniShotCut_website/)
12
+ [![GitHub](https://img.shields.io/badge/GitHub-Code-blue?logo=github&logoColor=white)](https://github.com/UVA-Computer-Vision-Lab/OmniShotCut)
13
  <a href="https://huggingface.co/spaces/uva-cv-lab/OmniShotCut"><img src="https://img.shields.io/static/v1?label=%F0%9F%A4%97%20HF%20Space&message=Online+Demo&color=orange"></a>
14
 
15
+ ## Installation 🔧
16
+ ```shell
17
+ conda create -n OmniShotCut python=3.10
18
+ conda activate OmniShotCut
19
+ pip install torch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 --index-url https://download.pytorch.org/whl/cu124
20
+ pip install -r requirements.txt
21
+ conda install ffmpeg
22
+ ```
23
+
24
+ First, download the checkpoint:
25
+ ```shell
26
+ mkdir checkpoints
27
+ cd checkpoints
28
+ wget https://huggingface.co/uva-cv-lab/OmniShotCut/resolve/main/OmniShotCut_ckpt.pth
29
+ ```
30
+
31
+ ## Inference ⚡
32
+ We provide several modes for inference. The `clean_shot` mode is recommended for users who want the most direct results (valid shots without any transitions and sudden jumps).
33
+
34
+ Execute the inference by running:
35
+ ```shell
36
+ python test_code/inference.py --checkpoint_path checkpoints/OmniShotCut_ckpt.pth --input_video_path path/to/your/video.mp4 --mode clean_shot
37
+ ```
38
+ The results will be visualized in a folder named `demo_video_results`, where vertical bars with the same color refer to the same shot.
39
+
40
+ ## Citation
41
+ ```bibtex
42
+ @article{wang2026omnishotcut,
43
+ title={OmniShotCut: Holistic Relational Shot Boundary Detection with Shot-Query Transformer},
44
+ author={Wang, Boyang and Xu, Guangyi and Tang, Zhipeng and Zhang, Jiahui and Cheng, Zezhou},
45
+ journal={arXiv preprint arXiv:2604.24762},
46
+ year={2026}
47
+ }
48
+ ```