preTrained predictions are ready for observation

Browse files

Files changed (3) hide show

README.md +1 -405
maskcut/README.md +1 -292
third_party/README.md +1 -293

README.md CHANGED Viewed

@@ -1,405 +1 @@
-# Cut and Learn for Unsupervised Image & Video Object Detection and Instance Segmentation
-**Cut**-and-**LE**a**R**n (**CutLER**) is a simple approach for training object detection and instance segmentation models without human annotations.
-It outperforms previous SOTA by **2.7 times** for AP50 and **2.6 times** for AR on **11 benchmarks**.
-<p align="center"> <img src='docs/teaser_img.jpg' align="center" > </p>
-> [**Cut and Learn for Unsupervised Object Detection and Instance Segmentation**](http://people.eecs.berkeley.edu/~xdwang/projects/CutLER/)
-> [Xudong Wang](https://people.eecs.berkeley.edu/~xdwang/), [Rohit Girdhar](https://rohitgirdhar.github.io/), [Stella X. Yu](https://www1.icsi.berkeley.edu/~stellayu/), [Ishan Misra](https://imisra.github.io/)
-> FAIR, Meta AI; UC Berkeley
-> CVPR 2023
-[[`project page`](http://people.eecs.berkeley.edu/~xdwang/projects/CutLER/)] [[`arxiv`](https://arxiv.org/abs/2301.11320)] [[`colab`](https://colab.research.google.com/drive/1NgEyFHvOfuA2MZZnfNPWg1w5gSr3HOBb?usp=sharing)] [[`bibtex`](#citation)]
-Unsupervised video instance segmentation (**VideoCutLER**) is also supported. ***We demonstrate that video instance segmentation models can be learned without using any human annotations, without relying on natural videos (ImageNet data alone is sufficient), and even without motion estimations!*** The code is available [here](videocutler).
-<p align="center">
-  <img src="docs/demos_videocutler.gif" width=100%>
-</p>
-> [**VideoCutLER: Surprisingly Simple Unsupervised Video Instance Segmentation**](https://people.eecs.berkeley.edu/~xdwang/projects/VideoCutLER/videocutler.pdf)
-> [Xudong Wang](https://people.eecs.berkeley.edu/~xdwang/), [Ishan Misra](https://imisra.github.io/), Ziyun Zeng, [Rohit Girdhar](https://rohitgirdhar.github.io/), [Trevor Darrell](https://people.eecs.berkeley.edu/~trevor/)
-> UC Berkeley; FAIR, Meta AI
-> CVPR 2024
-[[`code`](videocutler/README.md)] [[`PDF`](https://people.eecs.berkeley.edu/~xdwang/projects/VideoCutLER/videocutler.pdf)] [[`arxiv`](https://arxiv.org/abs/2308.14710)] [[`bibtex`](#citation)]
-## Features
-- We propose MaskCut approach to generate pseudo-masks for multiple objects in an image.
-- CutLER can learn unsupervised object detectors and instance segmentors solely on ImageNet-1K.
-- CutLER exhibits strong robustness to domain shifts when evaluated on 11 different benchmarks across domains like natural images, video frames, paintings, sketches, etc.
-- CutLER can serve as a pretrained model for fully/semi-supervised detection and segmentation tasks.
-- We also propose VideoCutLER, a surprisingly simple unsupervised video instance segmentation (UVIS) method without relying on optical flows. ImaegNet-1K is all we need for training a SOTA UVIS model!
-## Installation
-See [installation instructions](INSTALL.md).
-## Dataset Preparation
-See [Preparing Datasets for CutLER](datasets/README.md).
-## Method Overview
-<p align="center">
-  <img src="docs/pipeline.jpg" width=55%>
-</p>
-Cut-and-Learn has two stages: 1) generating pseudo-masks with MaskCut and 2) learning unsupervised detectors from pseudo-masks of unlabeled data.
-### 1. MaskCut
-MaskCut can be used to provide segmentation masks for multiple instances of each image.
-<p align="center">
-  <img src="docs/maskcut.gif" width=100%>
-</p>
-### MaskCut Demo
-Try out the MaskCut demo using Colab (no GPU needed): [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1X05lKL_IBRvZB7q6n6pb4w00_tIYjGlf?usp=sharing)
-Try out the web demo: [![Hugging Face Spaces](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-blue)](https://huggingface.co/spaces/facebook/MaskCut) (thanks to [@hysts](https://github.com/hysts)!)
-If you want to run MaskCut locally, we provide `demo.py` that is able to visualize the pseudo-masks produced by MaskCut.
-Run it with:
-```
-cd maskcut
-python demo.py --img-path imgs/demo2.jpg \
-  --N 3 --tau 0.15 --vit-arch base --patch-size 8 \
-  [--other-options]
-```
-We give a few demo images in maskcut/imgs/. If you want to run demo.py with cpu, simply add "--cpu" when running the demo script.
-For imgs/demo4.jpg, you need to use "--N 6" to segment all six instances in the image.
-Following, we give some visualizations of the pseudo-masks on the demo images.
-<p align="center">
-  <img src="docs/maskcut-demo.jpg" width=100%>
-</p>
-### Generating Annotations for ImageNet-1K with MaskCut
-To generate pseudo-masks for ImageNet-1K using MaskCut, first set up the ImageNet-1K dataset according to the instructions in [datasets/README.md](datasets/README.md), then execute the following command:
-```
-cd maskcut
-python maskcut.py \
---vit-arch base --patch-size 8 \
---tau 0.15 --fixed_size 480 --N 3 \
---num-folder-per-job 1000 --job-index 0 \
---dataset-path /path/to/dataset/traindir \
---out-dir /path/to/save/annotations \
-```
-As the process of generating pseudo-masks for all 1.3 million images in 1,000 folders takes a significant amount of time, it is recommended to use multiple runs. Each run should process the pseudo-mask generation for a smaller number of image folders by setting "--num-folder-per-job" and "--job-index". Once all runs are completed, you can merge all the resulting json files by using the following command:
-```
-python merge_jsons.py \
---base-dir /path/to/save/annotations \
---num-folder-per-job 2 --fixed-size 480 \
---tau 0.15 --N 3 \
---save-path imagenet_train_fixsize480_tau0.15_N3.json
-```
-The "--num-folder-per-job", "--fixed-size", "--tau" and "--N" of merge_jsons.py should match the ones used to run maskcut.py.
-We also provide a submitit script to launch the pseudo-mask generation process with multiple nodes.
-```
-cd maskcut
-bash run_maskcut_with_submitit.sh
-```
-After that, you can use "merge_jsons.py" to merge all these json files as described above.
-### 2. CutLER
-### Inference Demo for CutLER with Pre-trained Models
-Try out the CutLER demo using Colab (no GPU needed): [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1NgEyFHvOfuA2MZZnfNPWg1w5gSr3HOBb?usp=sharing)
-Try out the web demo: [![Hugging Face Spaces](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-blue)](https://huggingface.co/spaces/facebook/CutLER) (thanks to [@hysts](https://github.com/hysts)!)
-Try out Replicate demo and the API: [![Replicate](https://replicate.com/cjwbw/cutler/badge)](https://replicate.com/cjwbw/cutler)
-If you want to run CutLER demos locally,
-1. Pick a model and its config file from [model zoo](#model-zoo),
-  for example, `model_zoo/configs/CutLER-ImageNet/cascade_mask_rcnn_R_50_FPN.yaml`.
-2. We provide `demo.py` that is able to demo builtin configs. Run it with:
-```
-cd cutler
-python demo/demo.py --config-file model_zoo/configs/CutLER-ImageNet/cascade_mask_rcnn_R_50_FPN_demo.yaml \
-  --input demo/imgs/*.jpg \
-  [--other-options]
-  --opts MODEL.WEIGHTS /path/to/cutler_w_cascade_checkpoint
-```
-The configs are made for training, therefore we need to specify `MODEL.WEIGHTS` to a model from model zoo for evaluation.
-This command will run the inference and show visualizations in an OpenCV window.
-<!-- For details of the command line arguments, see `demo.py -h` or look at its source code
-to understand its behavior. Some common arguments are: -->
-* To run __on cpu__, add `MODEL.DEVICE cpu` after `--opts`.
-* To save outputs to a directory (for images) or a file (for webcam or video), use `--output`.
-Following, we give some visualizations of the model predictions on the demo images.
-<p align="center">
-  <img src="docs/cutler-demo.jpg" width=100%>
-</p>
-### Unsupervised Model Learning
-Before training the detector, it is necessary to use MaskCut to generate pseudo-masks for all ImageNet data.
-You can either use the pre-generated json file directly by downloading it from [here](http://dl.fbaipublicfiles.com/cutler/maskcut/imagenet_train_fixsize480_tau0.15_N3.json) and placing it under "DETECTRON2_DATASETS/imagenet/annotations/", or generate your own pseudo-masks by following the instructions in [MaskCut](#1-maskcut).
-We provide a script `train_net.py`, that is made to train all the configs provided in CutLER.
-To train a model with "train_net.py", first setup the ImageNet-1K dataset following [datasets/README.md](datasets/README.md), then run:
-```
-cd cutler
-export DETECTRON2_DATASETS=/path/to/DETECTRON2_DATASETS/
-python train_net.py --num-gpus 8 \
-  --config-file model_zoo/configs/CutLER-ImageNet/cascade_mask_rcnn_R_50_FPN.yaml
-```
-If you want to train a model using multiple nodes, you may need to adjust [some model parameters](https://arxiv.org/abs/1706.02677) and some SBATCH command options in "tools/train-1node.sh" and "tools/single-node_run.sh", then run:
-```
-cd cutler
-sbatch tools/train-1node.sh \
-  --config-file model_zoo/configs/CutLER-ImageNet/cascade_mask_rcnn_R_50_FPN.yaml \
-  MODEL.WEIGHTS /path/to/dino/d2format/model \
-  OUTPUT_DIR output/
-```
-You can also convert a pre-trained DINO model to detectron2's format by yourself following [this link](https://github.com/facebookresearch/moco/tree/main/detection).
-### Self-training
-We further improve performance by self-training the model on its predictions.
-Firstly, we can get model predictions on ImageNet via running:
-```
-python train_net.py --num-gpus 8 \
-  --config-file model_zoo/configs/CutLER-ImageNet/cascade_mask_rcnn_R_50_FPN.yaml \
-  --test-dataset imagenet_train \
-  --eval-only TEST.DETECTIONS_PER_IMAGE 30 \
-  MODEL.WEIGHTS output/model_final.pth \ # load previous stage/round checkpoints
-  OUTPUT_DIR output/ # path to save model predictions
-```
-Secondly, we can run the following command to generate the json file for the first round of self-training:
-```
-python tools/get_self_training_ann.py \
-  --new-pred output/inference/coco_instances_results.json \ # load model predictions
-  --prev-ann DETECTRON2_DATASETS/imagenet/annotations/imagenet_train_fixsize480_tau0.15_N3.json \ # path to the old annotation file.
-  --save-path DETECTRON2_DATASETS/imagenet/annotations/cutler_imagenet1k_train_r1.json \ # path to save a new annotation file.
-  --threshold 0.7
-```
-Finally, place "cutler_imagenet1k_train_r1.json" under "DETECTRON2_DATASETS/imagenet/annotations/", then launch the self-training process:
-```
-python train_net.py --num-gpus 8 \
-  --config-file model_zoo/configs/CutLER-ImageNet/cascade_mask_rcnn_R_50_FPN_self_train.yaml \
-  --train-dataset imagenet_train_r1 \
-  MODEL.WEIGHTS output/model_final.pth \ # load previous stage/round checkpoints
-  OUTPUT_DIR output/self-train-r1/ # path to save checkpoints
-```
-You can repeat the steps above to perform multiple rounds of self-training and adjust some arguments as needed (e.g., "--threshold" for round 1 and 2 can be set to 0.7 and 0.65, respectively; "--train-dataset" for round 1 and 2 can be set to "imagenet_train_r1" and "imagenet_train_r2", respectively; MODEL.WEIGHTS for round 1 and 2 should point to the previous stage/round checkpoints). Ensure that all annotation files are placed under DETECTRON2_DATASETS/imagenet/annotations/.
-Please ensure that "--train-dataset", json file names and locations match the ones specified in "cutler/data/datasets/builtin.py".
-Please refer to this [instruction](https://detectron2.readthedocs.io/en/latest/tutorials/datasets.html) for guidance on using custom datasets.
-You can also directly download the MODEL.WEIGHTS and annotations used for each round of self-training:
-<table><tbody>
-<!-- START TABLE -->
-<!-- TABLE BODY -->
-<!-- ROW: round 1 -->
-<tr><td align="center">round 1</td>
-<td align="center"><a href="http://dl.fbaipublicfiles.com/cutler/checkpoints/cutler_cascade_r1.pth">cutler_cascade_r1.pth</a></td>
-<td align="center"><a href="http://dl.fbaipublicfiles.com/cutler/maskcut/cutler_imagenet1k_train_r1.json">cutler_imagenet1k_train_r1.json</a></td>
-</tr>
-<!-- ROW: round 2 -->
-<tr><td align="center">round 2</td>
-<td align="center"><a href="http://dl.fbaipublicfiles.com/cutler/checkpoints/cutler_cascade_r2.pth">cutler_cascade_r2.pth</a></td>
-<td align="center"><a href="http://dl.fbaipublicfiles.com/cutler/maskcut/cutler_imagenet1k_train_r2.json">cutler_imagenet1k_train_r2.json</a></td>
-</tr>
-</tbody></table>
-### Unsupervised Zero-shot Evaluation
-To evaluate a model's performance on 11 different datasets, please refer to [datasets/README.md](datasets/README.md) for instructions on preparing the datasets. Next, select a model from the model zoo, specify the "model_weights", "config_file" and the path to "DETECTRON2_DATASETS" in `tools/eval.sh`, then run the script.
-```
-bash tools/eval.sh
-```
-### Model Zoo
-We show zero-shot unsupervised object detection performance (AP50&nbsp;|&nbsp;AR) on 11 different datasets spanning a variety of domains. ^: CutLER using Mask R-CNN as a detector; *: CutLER using Cascade Mask R-CNN as a detector.
-<table><tbody>
-<!-- START TABLE -->
-<!-- TABLE HEADER -->
-<th valign="bottom">Methods</th>
-<th valign="bottom">Models</th>
-<th valign="bottom">COCO</th>
-<th valign="bottom">COCO20K</th>
-<th valign="bottom">VOC</th>
-<th valign="bottom">LVIS</th>
-<th valign="bottom">UVO</th>
-<th valign="bottom">Clipart</th>
-<th valign="bottom">Comic</th>
-<th valign="bottom">Watercolor</th>
-<th valign="bottom">KITTI</th>
-<th valign="bottom">Objects365</th>
-<th valign="bottom">OpenImages</th>
-<!-- TABLE BODY -->
-</tr>
-<tr><td align="center">Prev. SOTA</td>
-<td valign="bottom">-</td>
-<td align="center">9.6&nbsp;|&nbsp;12.6</td>
-<td align="center">9.7&nbsp;|&nbsp;12.6</td>
-<td align="center">15.9&nbsp;|&nbsp;21.3</td>
-<td align="center">3.8&nbsp;|&nbsp;6.4</td>
-<td align="center">10.0&nbsp;|&nbsp;14.2</td>
-<td align="center">7.9&nbsp;|&nbsp;15.1</td>
-<td align="center">9.9&nbsp;|&nbsp;16.3</td>
-<td align="center">6.7&nbsp;|&nbsp;16.2</td>
-<td align="center">7.7&nbsp;|&nbsp;7.1</td>
-<td align="center">8.1&nbsp;|&nbsp;10.2</td>
-<td align="center">9.9&nbsp;|&nbsp;14.9</td>
-</tr>
-<!-- ROW: Box/Mask AP for CutLER -->
-</tr>
-<tr><td align="center">CutLER^</td>
-<td valign="bottom"><a href="http://dl.fbaipublicfiles.com/cutler/checkpoints/cutler_mrcnn_final.pth">download</a></td>
-<td align="center">21.1&nbsp;|&nbsp;29.6</td>
-<td align="center">21.6&nbsp;|&nbsp;30.0</td>
-<td align="center">36.6&nbsp;|&nbsp;41.0</td>
-<td align="center">7.7&nbsp;|&nbsp;18.7</td>
-<td align="center">29.8&nbsp;|&nbsp;38.4</td>
-<td align="center">20.9&nbsp;|&nbsp;38.5</td>
-<td align="center">31.2&nbsp;|&nbsp;37.1</td>
-<td align="center">37.3&nbsp;|&nbsp;39.9</td>
-<td align="center">15.3&nbsp;|&nbsp;25.4</td>
-<td align="center">19.5&nbsp;|&nbsp;30.0</td>
-<td align="center">17.1&nbsp;|&nbsp;26.4</td>
-</tr>
-<!-- ROW: Box/Mask AP for CutLER -->
-</tr>
-<tr><td align="center">CutLER*</td>
-<td valign="bottom"><a href="http://dl.fbaipublicfiles.com/cutler/checkpoints/cutler_cascade_final.pth">download</a></td>
-<td align="center">21.9&nbsp;|&nbsp;32.7</td>
-<td align="center">22.4&nbsp;|&nbsp;33.1</td>
-<td align="center">36.9&nbsp;|&nbsp;44.3</td>
-<td align="center">8.4&nbsp;|&nbsp;21.8</td>
-<td align="center">31.7&nbsp;|&nbsp;42.8</td>
-<td align="center">21.1&nbsp;|&nbsp;41.3</td>
-<td align="center">30.4&nbsp;|&nbsp;38.6</td>
-<td align="center">37.5&nbsp;|&nbsp;44.6</td>
-<td align="center">18.4&nbsp;|&nbsp;27.5</td>
-<td align="center">21.6&nbsp;|&nbsp;34.2</td>
-<td align="center">17.3&nbsp;|&nbsp;29.6</td>
-</tr>
-</tbody></table>
-## Semi-supervised and Fully-supervised Learning
-CutLER can also serve as a pretrained model for training fully supervised object detection and instance segmentation models and improves performance on COCO, including on few-shot benchmarks.
-### Training & Evaluation in Command Line
-You can find all the semi-supervised and fully-supervised learning configs provided in CutLER under `model_zoo/configs/COCO-Semisupervised`.
-To train a model using K% labels with `train_net.py`, first set up the COCO dataset according to [datasets/README.md](datasets/README.md) and specify K value in the config file, then run:
-```
-python train_net.py --num-gpus 8 \
-  --config-file model_zoo/configs/COCO-Semisupervised/cascade_mask_rcnn_R_50_FPN_{K}perc.yaml \
-  MODEL.WEIGHTS /path/to/cutler_pretrained_model
-```
-You can find all config files used to train supervised models under `model_zoo/configs/COCO-Semisupervised`.
-The configs are made for 8-GPU training. To train on 1 GPU, you may need to [change some parameters](https://arxiv.org/abs/1706.02677), e.g. number of GPUs (num-gpus your_num_gpus), learning rates (SOLVER.BASE_LR your_base_lr) and batch size (SOLVER.IMS_PER_BATCH your_batch_size).
-### Evaluation
-To evaluate a model's performance, use
-```
-python train_net.py \
-  --config-file model_zoo/configs/COCO-Semisupervised/cascade_mask_rcnn_R_50_FPN_{K}perc.yaml \
-  --eval-only MODEL.WEIGHTS /path/to/checkpoint_file
-```
-For more options, see `python train_net.py -h`.
-### Model Zoo
-We fine-tune a Cascade R-CNN model initialized with CutLER or MoCo-v2 on varying amounts of labeled COCO data, and show results (Box&nbsp;|&nbsp;Mask AP) on the val2017 split below:
-<table><tbody>
-<!-- START TABLE -->
-<!-- TABLE HEADER -->
-<th valign="bottom">% of labels</th>
-<th valign="bottom">1%</th>
-<th valign="bottom">2%</th>
-<th valign="bottom">5%</th>
-<th valign="bottom">10%</th>
-<th valign="bottom">20%</th>
-<th valign="bottom">30%</th>
-<th valign="bottom">40%</th>
-<th valign="bottom">50%</th>
-<th valign="bottom">60%</th>
-<th valign="bottom">80%</th>
-<th valign="bottom">100%</th>
-<!-- TABLE BODY -->
-<!-- ROW: Box/Mask AP for CutLER -->
-<tr><td align="center">MoCo-v2</td>
-<td align="center">11.8&nbsp;|&nbsp;10.0</td>
-<td align="center">16.2&nbsp;|&nbsp;13.8</td>
-<td align="center">20.5&nbsp;|&nbsp;17.8</td>
-<td align="center">26.5&nbsp;|&nbsp;23.0</td>
-<td align="center">32.5&nbsp;|&nbsp;28.2</td>
-<td align="center">35.5&nbsp;|&nbsp;30.8</td>
-<td align="center">37.3&nbsp;|&nbsp;32.3</td>
-<td align="center">38.7&nbsp;|&nbsp;33.6</td>
-<td align="center">39.9&nbsp;|&nbsp;34.6</td>
-<td align="center">41.6&nbsp;|&nbsp;36.0</td>
-<td align="center">42.8&nbsp;|&nbsp;37.0</td>
-</tr>
-<!-- ROW: Mask AP -->
-<tr><td align="center">CutLER</td>
-<td align="center">16.8&nbsp;|&nbsp;14.6</td>
-<td align="center">21.6&nbsp;|&nbsp;18.9</td>
-<td align="center">27.8&nbsp;|&nbsp;24.3</td>
-<td align="center">32.2&nbsp;|&nbsp;28.1</td>
-<td align="center">36.6&nbsp;|&nbsp;31.7</td>
-<td align="center">38.2&nbsp;|&nbsp;33.3</td>
-<td align="center">39.9&nbsp;|&nbsp;34.7</td>
-<td align="center">41.5&nbsp;|&nbsp;35.9</td>
-<td align="center">42.3&nbsp;|&nbsp;36.7</td>
-<td align="center">43.8&nbsp;|&nbsp;37.9</td>
-<td align="center">44.7&nbsp;|&nbsp;38.5</td>
-</tr>
-<!-- ROW: Model Downloads -->
-<tr><td align="center">Download</td>
-<td align="center"><a href="http://dl.fbaipublicfiles.com/cutler/checkpoints/cutler_semi_1perc.pth">model</a></td>
-<td align="center"><a href="http://dl.fbaipublicfiles.com/cutler/checkpoints/cutler_semi_2perc.pth">model</a></td>
-<td align="center"><a href="http://dl.fbaipublicfiles.com/cutler/checkpoints/cutler_semi_5perc.pth">model</a></td>
-<td align="center"><a href="http://dl.fbaipublicfiles.com/cutler/checkpoints/cutler_semi_10perc.pth">model</a></td>
-<td align="center"><a href="http://dl.fbaipublicfiles.com/cutler/checkpoints/cutler_semi_20perc.pth">model</a></td>
-<td align="center"><a href="http://dl.fbaipublicfiles.com/cutler/checkpoints/cutler_semi_30perc.pth">model</a></td>
-<td align="center"><a href="http://dl.fbaipublicfiles.com/cutler/checkpoints/cutler_semi_40perc.pth">model</a></td>
-<td align="center"><a href="http://dl.fbaipublicfiles.com/cutler/checkpoints/cutler_semi_50perc.pth">model</a></td>
-<td align="center"><a href="http://dl.fbaipublicfiles.com/cutler/checkpoints/cutler_semi_60perc.pth">model</a></td>
-<td align="center"><a href="http://dl.fbaipublicfiles.com/cutler/checkpoints/cutler_semi_80perc.pth">model</a></td>
-<td align="center"><a href="http://dl.fbaipublicfiles.com/cutler/checkpoints/cutler_fully_100perc.pth">model</a></td>
-</tr>
-</tbody></table>
-Both MoCo-v2 and our CutLER are trained for the 1x schedule using Detectron2, except for extremely low-shot settings with 1% or 2% labels. When training with 1% or 2% labels, we train both MoCo-v2 and our model for 3,600 iterations with a batch size of 16.
-## License
-The majority of CutLER, Detectron2 and DINO are licensed under the [CC-BY-NC license](LICENSE), however portions of the project are available under separate license terms: TokenCut, Bilateral Solver and CRF are licensed under the MIT license; If you later add other third party code, please keep this license info updated, and please let us know if that component is licensed under something other than CC-BY-NC, MIT, or CC0.
-## Ethical Considerations
-CutLER's wide range of detection capabilities may introduce similar challenges to many other visual recognition methods.
-As the image can contain arbitrary instances, it may impact the model output.
-## How to get support from us?
-If you have any general questions, feel free to email us at [Xudong Wang](mailto:xdwang@eecs.berkeley.edu), [Ishan Misra](mailto:imisra@meta.com) and [Rohit Girdhar](mailto:rgirdhar@meta.com). If you have code or implementation-related questions, please feel free to send emails to us or open an issue in this codebase (We recommend that you open an issue in this codebase, because your questions may help others).
-## Citation
-If you find our work inspiring or use our codebase in your research, please consider giving a star ⭐ and a citation.
-```
-@inproceedings{wang2023cut,
-  title={Cut and learn for unsupervised object detection and instance segmentation},
-  author={Wang, Xudong and Girdhar, Rohit and Yu, Stella X and Misra, Ishan},
-  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
-  pages={3124--3134},
-  year={2023}
-}
-```
-```
-@article{wang2023videocutler,
-  title={VideoCutLER: Surprisingly Simple Unsupervised Video Instance Segmentation},
-  author={Wang, Xudong and Misra, Ishan and Zeng, Ziyun and Girdhar, Rohit and Darrell, Trevor},
-  journal={arXiv preprint arXiv:2308.14710},
-  year={2023}
-}
-```


1	+ Under Development for research

maskcut/README.md CHANGED Viewed

@@ -1,293 +1,2 @@
-# (CVPR 2022) TokenCut
-Pytorch implementation of **Tokencut**:
-**Self-supervised Transformers for Unsupervised Object Discovery using Normalized Cut**
-*[Yangtao Wang](https://yangtaowang95.github.io), [Xi Shen](https://xishen0220.github.io/), [Shell Xu Hu](http://hushell.github.io/), [Yuan Yuan](https://yyuanad.github.io/), [James L. Crowley](http://crowley-coutaz.fr/jlc/jlc.html), [Dominique Vaufreydaz](https://research.vaufreydaz.org/)*
-[[Project page](https://www.m-psi.fr/Papers/TokenCut2022/)]
-[[ Github (Video Segmentation) ](https://github.com/YangtaoWANG95/TokenCut_video)]
-[[Paper](https://arxiv.org/pdf/2202.11539.pdf)]
-[![Colab demo](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/YangtaoWANG95/TokenCut/blob/master/inference_demo.ipynb)
-[![Hugging Face Spaces](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-blue)](https://huggingface.co/spaces/yangtaowang/TokenCut)
-<p align="center">
-  <img width="100%" alt="TokenCut teaser" src="examples/overview.png">
-</p>
-If our project is helpful for your research, please consider citing :
-```
-@inproceedings{wang2022tokencut,
-          title={Self-supervised Transformers for Unsupervised Object Discovery using Normalized Cut},
-          author={Wang, Yangtao and Shen, Xi and Hu, Shell Xu and Yuan, Yuan and Crowley, James L. and Vaufreydaz, Dominique},
-          booktitle={Conference on Computer Vision and Pattern Recognition}
-          year={2022}
-        }
-```
-## Table of Content
-* [1. Updates](#1-updates)
-* [2. Installation](#2-installation)
-  * [2.1 Dependencies](#21-dependencies)
-  * [2.2 Data](#22-data)
-* [3. Quick Start](#3-quick-start)
-  * [3.1 Detecting an object in one image](#31-detecting-an-object-in-one-image)
-  * [3.2 Segmenting a salient region in one image](#32-segmenting-a-salient-region-in-one-image)
-* [4. Evaluation](#4-evaluation)
-  * [4.1 Unsupervised object discovery](#41-unsupervised-object-discovery)
-  * [4.2 Unsupervised saliency detection](#42-unsupervised-saliency-detection)
-  * [4.3 Weakly supervised object detection](#43-weakly-supervised-object-detection)
-* [5. Acknowledgement](#5-acknowledgement)
-## 1. Updates
-***09/06/2022***
-Extension work of [TokeCut Video Segmentation](https://github.com/YangtaoWANG95/TokenCut_video) is realised!
-***03/10/2022***
-Creating a 480p Demo using [Gradio](https://github.com/gradio-app/gradio). Try out the Web Demo: [![Hugging Face Spaces](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-blue)](https://huggingface.co/spaces/yangtaowang/TokenCut)
-Internet image results:
-<p >
-  <img width="20%" alt="TokenCut visualizations" src="examples/internet_image/kungfu_pred.jpg">
-  <img width="20%" alt="TokenCut visualizations" src="examples/internet_image/kungfu_attn.jpg">
-  <img width="19%" alt="TokenCut visualizations" src="examples/internet_image/pokemon_pred.jpg">
-  <img width="19.5%" alt="TokenCut visualizations" src="examples/internet_image/pokemon_attn.jpg">
-</p>
-***02/26/2022***
-Integrated into [Huggingface Spaces 🤗](https://huggingface.co/spaces) using [Gradio](https://github.com/gradio-app/gradio). Try out the Web Demo: [![Hugging Face Spaces](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-blue)](https://huggingface.co/spaces/akhaliq/TokenCut)
-***02/26/2022***
-A simple TokenCut Colab Demo is available.
-***02/21/2022***
-Initial commit: Code of TokenCut is released, including evaluation of unsupervised object discovery, unsupervised saliency object detection, weakly supervised object locolization.
-## 2. Installation
-### 2.1 Dependencies
-This code was implemented with Python 3.7, PyTorch 1.7.1 and CUDA 11.2. Please refer to [the official installation](https://pytorch.org/get-started/previous-versions/). If CUDA 10.2 has been properly installed :
-```
-pip install torch==1.7.1 torchvision==0.8.2
-```
-In order to install the additionnal dependencies, please launch the following command:
-```
-pip install -r requirements.txt
-```
-### 2.2 Data
-We provide quick download commands in [DOWNLOAD_DATA.md](./DOWNLOAD_DATA.md) for VOC2007, VOC2012, COCO, CUB, ImageNet, ECSSD, DUTS and DUT-OMRON as well as DINO checkpoints.
-## 3. Quick Start
-### 3.1 Detecting an object in one image
-We provide TokenCut visualization for bounding box prediction and attention map. Using `all` for all visualization results.
-<!--Following are scripts to apply TokenCut to an image defined via the `image_path` parameter and visualize the predictions (`pred`), the eigen attention map of the Figure 3 in the paper (`attn`), and all the figures(`all`). Box predictions are also stored in the output directory given by parameter `output_dir`. -->
-```
-python main_tokencut.py --image_path examples/VOC07_000036.jpg --visualize pred
-python main_tokencut.py --image_path examples/VOC07_000036.jpg --visualize attn
-python main_tokencut.py --image_path examples/VOC07_000036.jpg --visualize all
-```
-### 3.2 Segmenting a salient region in one image
-We provide TokenCut segmentation results as follows:
-```
-cd unsupervised_saliency_detection
-python get_saliency.py --sigma-spatial 16 --sigma-luma 16 --sigma-chroma 8 --vit-arch small --patch-size 16 --img-path ../examples/VOC07_000036.jpg --out-dir ./output
-```
-## 4. Evaluation
-Following are the different steps to reproduce the results of **TokenCut** presented in the paper.
-### 4.1 Unsupervised object discovery
-<p align="center">
-  <img width="30%" alt="TokenCut visualizations" src="examples/ex000012.jpg">
-  <img width="13.2%" alt="TokenCut visualizations" src="examples/ex000036.jpg">
-  <img width="19.8%" alt="TokenCut visualizations" src="examples/ex000064.jpg">
-</p>
-#### PASCAL-VOC
-In order to apply TokenCut and compute corloc results (VOC07 68.8, VOC12 72.1), please launch:
-```
-python main_tokencut.py --dataset VOC07 --set trainval
-python main_tokencut.py --dataset VOC12 --set trainval
-```
-If you want to extract Dino features, which corresponds to [the KEY features in DINO](https://github.com/XiSHEN0220/LOST/blob/main/main_lost.py#L259-L261):
-```
-mkdir features
-python main_lost.py --dataset VOC07 --set trainval --save-feat-dir features/VOC2007
-```
-#### COCO
-Results are provided given the 2014 annotations following previous works. The following command line allows you to get results on the subset of 20k images of the COCO dataset (corloc 58.8), following previous litterature. To be noted that the 20k images are a subset of the `train` set.
-```
-python main_tokencut.py --dataset COCO20k --set train
-```
-#### Different models
-We have tested the method on different setups of the VIT model, corloc results are presented in the following table (more can be found in the paper).
-<table>
-  <tr>
-    <th>arch</th>
-    <th>pre-training</th>
-    <th colspan="3">dataset</th>
-  </tr>
-  <tr>
-    <th></th>
-    <th></th>
-    <th>VOC07</th>
-    <th>VOC12</th>
-    <th>COCO20k</th>
-  </tr>
-  <tr>
-    <td>ViT-S/16</td>
-    <td>DINO</td>
-    <td>68.8</td>
-    <td>72.1</td>
-    <td>58.8</td>
-  <tr>
-  <tr>
-    <td>ViT-S/8</td>
-    <td>DINO</td>
-    <td>67.3</td>
-    <td>71.6</td>
-    <td>60.7</td>
-  <tr>
-  <tr>
-    <td>ViT-B/16</td>
-    <td>DINO</td>
-    <td>68.8</td>
-    <td>72.4</td>
-    <td>59.0</td>
-  <tr>
-</table>
-Previous results on the dataset `VOC07` can be obtained by launching:
-```
-python main_tokencut.py --dataset VOC07 --set trainval #VIT-S/16
-python main_tokencut.py --dataset VOC07 --set trainval --patch_size 8 #VIT-S/8
-python main_tokencut.py --dataset VOC07 --set trainval --arch vit_base #VIT-B/16
-```
-### 4.2 Unsupervised saliency detection
-<p align="center">
-  <img width="28%" alt="TokenCut visualizations" src="examples/ex_usd_2.jpg">
-  <img width="12.5%" alt="TokenCut visualizations" src="examples/ex_usd_3.jpg">
-  <img width="14%" alt="TokenCut visualizations" src="examples/ex_usd_1.jpg">
-</p>
-To evaluate on ECSSD, DUTS, DUT_OMRON dataset:
-```
-python get_saliency.py --out-dir ECSSD --sigma-spatial 16 --sigma-luma 16 --sigma-chroma 8 --nb-vis 1 --vit-arch small --patch-size 16 --dataset ECSSD
-python get_saliency.py --out-dir DUTS --sigma-spatial 16 --sigma-luma 16 --sigma-chroma 8 --nb-vis 1 --vit-arch small --patch-size 16 --dataset DUTS
-python get_saliency.py --out-dir DUT --sigma-spatial 16 --sigma-luma 16 --sigma-chroma 8 --nb-vis 1 --vit-arch small --patch-size 16 --dataset DUT
-```
-This should give:
-<table>
-  <tr>
-    <th>Method</th>
-    <th colspan="3"> ECSSD</th>
-    <th colspan="3"> DUTS</th>
-    <th colspan="3"> DUT-OMRON</th>
-  </tr>
-  <tr>
-    <th></th>
-    <th>maxF</th>
-    <th>IoU</th>
-    <th>Acc</th>
-    <th>maxF</th>
-    <th>IoU</th>
-    <th>Acc</th>
-    <th>maxF</th>
-    <th>IoU</th>
-    <th>Acc</th>
-  </tr>
-  <tr>
-    <td>TokenCut</td>
-    <td>80.3</td>
-    <td>71.2</td>
-    <td>91.8</td>
-    <td>67.2</td>
-    <td>57.6</td>
-    <td>90.3</td>
-    <td>60.0</td>
-    <td>53.3</td>
-    <td>88.0</td>
-  <tr>
-  <tr>
-    <td>TokenCut + BS</td>
-    <td>87.4</td>
-    <td>77.2</td>
-    <td>93.4</td>
-    <td>75.5</td>
-    <td>62,4</td>
-    <td>91.4</td>
-    <td>69.7</td>
-    <td>61.8</td>
-    <td>89.7</td>
-  </tr>
-</table>
-### 4.3 Weakly supervised object detection
-<p align="center">
-  <img width="15%" alt="TokenCut visualizations" src="examples/ex_wsd_1.jpg">
-  <img width="24%" alt="TokenCut visualizations" src="examples/ex_wsd_2.jpg">
-  <img width="18.9%" alt="TokenCut visualizations" src="examples/ex_wsd_3.jpg">
-</p>
-#### Fintune DINO small on CUB
-To finetune ViT-S/16 on CUB on a single node with 4 gpus for 1000 epochs run:
-```
-python -m torch.distributed.launch --nproc_per_node=4 main.py --data_path /path/to/data --batch_size_per_gpu 256 --dataset cub --weight_decay 0.005 --pretrained_weights ./dino_deitsmall16_pretrain.pth --epoch 1000 --output_dir ./path/to/checkpoin --lr 2e-4 --warmup-epochs 50 --num_labels 200 --num_workers 16 --n_last_blocks 1 --avgpool_patchtokens true --arch vit_small --patch_size 16
-```
-#### Evaluation on CUB
-To evaluate a fine-tuned ViT-S/16 on CUB val with a single GPU run:
-```
-python eval.py --pretrained_weights ./path/to/checkpoint --dataset cub --data_path ./path/to/data --batch_size_per_gpu 1 --no_center_crop
-```
-This should give:
-```
-Top1_cls: 79.12, top5_cls94.80, gt_loc: 0.914, top1_loc:0.723
-```
-#### Evaluate on Imagenet
-To Evaluate ViT-S/16 finetuned on ImageNet val with a single GPU run:
-```
-python eval.py --pretrained_weights /path/to/checkpoint --classifier_weights /path/to/linear_weights--dataset imagenet --data_path ./path/to/data --batch_size_per_gpu 1 --num_labels 1000 --batch_size_per_gpu 1 --no_center_crop --input_size 256 --tau 0.2 --patch_size 16 --arch vit_small
-```
-## 5. Acknowledgement
-TokenCut code is built on top of [LOST](https://github.com/valeoai/LOST), [DINO](https://github.com/facebookresearch/dino), [Segswap](https://github.com/XiSHEN0220/SegSwap), and [Bilateral_Sovlver](https://github.com/poolio/bilateral_solver). We would like to sincerely thanks those authors for their great works.






































































































































































































































































































1
2	+ Self-supervised Transformers for Unsupervised Object Discovery using Normalized Cut

third_party/README.md CHANGED Viewed

@@ -1,293 +1 @@
-# (CVPR 2022) TokenCut
-Pytorch implementation of **Tokencut**:
-**Self-supervised Transformers for Unsupervised Object Discovery using Normalized Cut**
-*[Yangtao Wang](https://yangtaowang95.github.io), [Xi Shen](https://xishen0220.github.io/), [Shell Xu Hu](http://hushell.github.io/), [Yuan Yuan](https://yyuanad.github.io/), [James L. Crowley](http://crowley-coutaz.fr/jlc/jlc.html), [Dominique Vaufreydaz](https://research.vaufreydaz.org/)*
-[[Project page](https://www.m-psi.fr/Papers/TokenCut2022/)]
-[[ Github (Video Segmentation) ](https://github.com/YangtaoWANG95/TokenCut_video)]
-[[Paper](https://arxiv.org/pdf/2202.11539.pdf)]
-[![Colab demo](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/YangtaoWANG95/TokenCut/blob/master/inference_demo.ipynb)
-[![Hugging Face Spaces](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-blue)](https://huggingface.co/spaces/yangtaowang/TokenCut)
-<p align="center">
-  <img width="100%" alt="TokenCut teaser" src="examples/overview.png">
-</p>
-If our project is helpful for your research, please consider citing :
-```
-@inproceedings{wang2022tokencut,
-          title={Self-supervised Transformers for Unsupervised Object Discovery using Normalized Cut},
-          author={Wang, Yangtao and Shen, Xi and Hu, Shell Xu and Yuan, Yuan and Crowley, James L. and Vaufreydaz, Dominique},
-          booktitle={Conference on Computer Vision and Pattern Recognition}
-          year={2022}
-        }
-```
-## Table of Content
-* [1. Updates](#1-updates)
-* [2. Installation](#2-installation)
-  * [2.1 Dependencies](#21-dependencies)
-  * [2.2 Data](#22-data)
-* [3. Quick Start](#3-quick-start)
-  * [3.1 Detecting an object in one image](#31-detecting-an-object-in-one-image)
-  * [3.2 Segmenting a salient region in one image](#32-segmenting-a-salient-region-in-one-image)
-* [4. Evaluation](#4-evaluation)
-  * [4.1 Unsupervised object discovery](#41-unsupervised-object-discovery)
-  * [4.2 Unsupervised saliency detection](#42-unsupervised-saliency-detection)
-  * [4.3 Weakly supervised object detection](#43-weakly-supervised-object-detection)
-* [5. Acknowledgement](#5-acknowledgement)
-## 1. Updates
-***09/06/2022***
-Extension work of [TokeCut Video Segmentation](https://github.com/YangtaoWANG95/TokenCut_video) is realised!
-***03/10/2022***
-Creating a 480p Demo using [Gradio](https://github.com/gradio-app/gradio). Try out the Web Demo: [![Hugging Face Spaces](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-blue)](https://huggingface.co/spaces/yangtaowang/TokenCut)
-Internet image results:
-<p >
-  <img width="20%" alt="TokenCut visualizations" src="examples/internet_image/kungfu_pred.jpg">
-  <img width="20%" alt="TokenCut visualizations" src="examples/internet_image/kungfu_attn.jpg">
-  <img width="19%" alt="TokenCut visualizations" src="examples/internet_image/pokemon_pred.jpg">
-  <img width="19.5%" alt="TokenCut visualizations" src="examples/internet_image/pokemon_attn.jpg">
-</p>
-***02/26/2022***
-Integrated into [Huggingface Spaces 🤗](https://huggingface.co/spaces) using [Gradio](https://github.com/gradio-app/gradio). Try out the Web Demo: [![Hugging Face Spaces](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Spaces-blue)](https://huggingface.co/spaces/akhaliq/TokenCut)
-***02/26/2022***
-A simple TokenCut Colab Demo is available.
-***02/21/2022***
-Initial commit: Code of TokenCut is released, including evaluation of unsupervised object discovery, unsupervised saliency object detection, weakly supervised object locolization.
-## 2. Installation
-### 2.1 Dependencies
-This code was implemented with Python 3.7, PyTorch 1.7.1 and CUDA 11.2. Please refer to [the official installation](https://pytorch.org/get-started/previous-versions/). If CUDA 10.2 has been properly installed :
-```
-pip install torch==1.7.1 torchvision==0.8.2
-```
-In order to install the additionnal dependencies, please launch the following command:
-```
-pip install -r requirements.txt
-```
-### 2.2 Data
-We provide quick download commands in [DOWNLOAD_DATA.md](./DOWNLOAD_DATA.md) for VOC2007, VOC2012, COCO, CUB, ImageNet, ECSSD, DUTS and DUT-OMRON as well as DINO checkpoints.
-## 3. Quick Start
-### 3.1 Detecting an object in one image
-We provide TokenCut visualization for bounding box prediction and attention map. Using `all` for all visualization results.
-<!--Following are scripts to apply TokenCut to an image defined via the `image_path` parameter and visualize the predictions (`pred`), the eigen attention map of the Figure 3 in the paper (`attn`), and all the figures(`all`). Box predictions are also stored in the output directory given by parameter `output_dir`. -->
-```
-python main_tokencut.py --image_path examples/VOC07_000036.jpg --visualize pred
-python main_tokencut.py --image_path examples/VOC07_000036.jpg --visualize attn
-python main_tokencut.py --image_path examples/VOC07_000036.jpg --visualize all
-```
-### 3.2 Segmenting a salient region in one image
-We provide TokenCut segmentation results as follows:
-```
-cd unsupervised_saliency_detection
-python get_saliency.py --sigma-spatial 16 --sigma-luma 16 --sigma-chroma 8 --vit-arch small --patch-size 16 --img-path ../examples/VOC07_000036.jpg --out-dir ./output
-```
-## 4. Evaluation
-Following are the different steps to reproduce the results of **TokenCut** presented in the paper.
-### 4.1 Unsupervised object discovery
-<p align="center">
-  <img width="30%" alt="TokenCut visualizations" src="examples/ex000012.jpg">
-  <img width="13.2%" alt="TokenCut visualizations" src="examples/ex000036.jpg">
-  <img width="19.8%" alt="TokenCut visualizations" src="examples/ex000064.jpg">
-</p>
-#### PASCAL-VOC
-In order to apply TokenCut and compute corloc results (VOC07 68.8, VOC12 72.1), please launch:
-```
-python main_tokencut.py --dataset VOC07 --set trainval
-python main_tokencut.py --dataset VOC12 --set trainval
-```
-If you want to extract Dino features, which corresponds to [the KEY features in DINO](https://github.com/XiSHEN0220/LOST/blob/main/main_lost.py#L259-L261):
-```
-mkdir features
-python main_lost.py --dataset VOC07 --set trainval --save-feat-dir features/VOC2007
-```
-#### COCO
-Results are provided given the 2014 annotations following previous works. The following command line allows you to get results on the subset of 20k images of the COCO dataset (corloc 58.8), following previous litterature. To be noted that the 20k images are a subset of the `train` set.
-```
-python main_tokencut.py --dataset COCO20k --set train
-```
-#### Different models
-We have tested the method on different setups of the VIT model, corloc results are presented in the following table (more can be found in the paper).
-<table>
-  <tr>
-    <th>arch</th>
-    <th>pre-training</th>
-    <th colspan="3">dataset</th>
-  </tr>
-  <tr>
-    <th></th>
-    <th></th>
-    <th>VOC07</th>
-    <th>VOC12</th>
-    <th>COCO20k</th>
-  </tr>
-  <tr>
-    <td>ViT-S/16</td>
-    <td>DINO</td>
-    <td>68.8</td>
-    <td>72.1</td>
-    <td>58.8</td>
-  <tr>
-  <tr>
-    <td>ViT-S/8</td>
-    <td>DINO</td>
-    <td>67.3</td>
-    <td>71.6</td>
-    <td>60.7</td>
-  <tr>
-  <tr>
-    <td>ViT-B/16</td>
-    <td>DINO</td>
-    <td>68.8</td>
-    <td>72.4</td>
-    <td>59.0</td>
-  <tr>
-</table>
-Previous results on the dataset `VOC07` can be obtained by launching:
-```
-python main_tokencut.py --dataset VOC07 --set trainval #VIT-S/16
-python main_tokencut.py --dataset VOC07 --set trainval --patch_size 8 #VIT-S/8
-python main_tokencut.py --dataset VOC07 --set trainval --arch vit_base #VIT-B/16
-```
-### 4.2 Unsupervised saliency detection
-<p align="center">
-  <img width="28%" alt="TokenCut visualizations" src="examples/ex_usd_2.jpg">
-  <img width="12.5%" alt="TokenCut visualizations" src="examples/ex_usd_3.jpg">
-  <img width="14%" alt="TokenCut visualizations" src="examples/ex_usd_1.jpg">
-</p>
-To evaluate on ECSSD, DUTS, DUT_OMRON dataset:
-```
-python get_saliency.py --out-dir ECSSD --sigma-spatial 16 --sigma-luma 16 --sigma-chroma 8 --nb-vis 1 --vit-arch small --patch-size 16 --dataset ECSSD
-python get_saliency.py --out-dir DUTS --sigma-spatial 16 --sigma-luma 16 --sigma-chroma 8 --nb-vis 1 --vit-arch small --patch-size 16 --dataset DUTS
-python get_saliency.py --out-dir DUT --sigma-spatial 16 --sigma-luma 16 --sigma-chroma 8 --nb-vis 1 --vit-arch small --patch-size 16 --dataset DUT
-```
-This should give:
-<table>
-  <tr>
-    <th>Method</th>
-    <th colspan="3"> ECSSD</th>
-    <th colspan="3"> DUTS</th>
-    <th colspan="3"> DUT-OMRON</th>
-  </tr>
-  <tr>
-    <th></th>
-    <th>maxF</th>
-    <th>IoU</th>
-    <th>Acc</th>
-    <th>maxF</th>
-    <th>IoU</th>
-    <th>Acc</th>
-    <th>maxF</th>
-    <th>IoU</th>
-    <th>Acc</th>
-  </tr>
-  <tr>
-    <td>TokenCut</td>
-    <td>80.3</td>
-    <td>71.2</td>
-    <td>91.8</td>
-    <td>67.2</td>
-    <td>57.6</td>
-    <td>90.3</td>
-    <td>60.0</td>
-    <td>53.3</td>
-    <td>88.0</td>
-  <tr>
-  <tr>
-    <td>TokenCut + BS</td>
-    <td>87.4</td>
-    <td>77.2</td>
-    <td>93.4</td>
-    <td>75.5</td>
-    <td>62,4</td>
-    <td>91.4</td>
-    <td>69.7</td>
-    <td>61.8</td>
-    <td>89.7</td>
-  </tr>
-</table>
-### 4.3 Weakly supervised object detection
-<p align="center">
-  <img width="15%" alt="TokenCut visualizations" src="examples/ex_wsd_1.jpg">
-  <img width="24%" alt="TokenCut visualizations" src="examples/ex_wsd_2.jpg">
-  <img width="18.9%" alt="TokenCut visualizations" src="examples/ex_wsd_3.jpg">
-</p>
-#### Fintune DINO small on CUB
-To finetune ViT-S/16 on CUB on a single node with 4 gpus for 1000 epochs run:
-```
-python -m torch.distributed.launch --nproc_per_node=4 main.py --data_path /path/to/data --batch_size_per_gpu 256 --dataset cub --weight_decay 0.005 --pretrained_weights ./dino_deitsmall16_pretrain.pth --epoch 1000 --output_dir ./path/to/checkpoin --lr 2e-4 --warmup-epochs 50 --num_labels 200 --num_workers 16 --n_last_blocks 1 --avgpool_patchtokens true --arch vit_small --patch_size 16
-```
-#### Evaluation on CUB
-To evaluate a fine-tuned ViT-S/16 on CUB val with a single GPU run:
-```
-python eval.py --pretrained_weights ./path/to/checkpoint --dataset cub --data_path ./path/to/data --batch_size_per_gpu 1 --no_center_crop
-```
-This should give:
-```
-Top1_cls: 79.12, top5_cls94.80, gt_loc: 0.914, top1_loc:0.723
-```
-#### Evaluate on Imagenet
-To Evaluate ViT-S/16 finetuned on ImageNet val with a single GPU run:
-```
-python eval.py --pretrained_weights /path/to/checkpoint --classifier_weights /path/to/linear_weights--dataset imagenet --data_path ./path/to/data --batch_size_per_gpu 1 --num_labels 1000 --batch_size_per_gpu 1 --no_center_crop --input_size 256 --tau 0.2 --patch_size 16 --arch vit_small
-```
-## 5. Acknowledgement
-TokenCut code is built on top of [LOST](https://github.com/valeoai/LOST), [DINO](https://github.com/facebookresearch/dino), [Segswap](https://github.com/XiSHEN0220/SegSwap), and [Bilateral_Sovlver](https://github.com/poolio/bilateral_solver). We would like to sincerely thanks those authors for their great works.


1	+ Self-supervised Transformers for Unsupervised Object Discovery using Normalized Cut