Deep Common Feature Mining for Efficient Video Semantic Segmentation
This repository is the official implementation of Deep Common Feature Mining for Efficient Video Semantic Segmentation.
Please follow the guidelines in mmsegmentation
Requirements:
python == 3.8
pytorch >= 2.0.0
mmcv == 1.7.1
mmsegmentation == 0.30.0
tensorboardX
tqdm
yacs
ftfy
regex
timm
scikit-learn
Please follow VSPW to download VSPW 480P dataset.
Please follow Cityscapes to download Cityscapes dataset.
Please follow CamVid to download CamVid dataset.
After correctly downloading, the file system is as follows:
<dataset path>
├── VSPW_480p
│ ├── train.txt
| ├── val.txt
| ├── test.txt
| └── data
│ ├── video1
│ ├── video2
│ ├── ...
│
├── cityscapes_video
│ ├── gtFine
│ │ ├── train
│ │ └── val
│ └── leftImg8bit_sequence
│ ├── train
│ └── val
├── camvid
│ ├── 0001TP
│ ├── 0006R0
│ ├── 0016E5
│ ├── Seq05VD
│ ├── gt
│ ├── test.txt
│ ├── train.txt
│ ├── val.txt
The dataset should be put in DCFM/data. Or you can use Symlink:
cd DCFM
ln -s <dataset path> ./data
Download weights (google drive|onedrive) pretrained on ImageNet-1K (provided by SegFormer), and put them in a folder pretrained/.
Download weights (resnet18, resnet50&101) of ResNet.
Place the .pth files in DCFM/pretrain directory.
torchrun --nproc_per_node=4 --master_port=<port> ./tools/train.py --root ./exp/vspw/dcfm/mitb2 --gpus 0,1,2,3The checkpoints and output information generated during training will be saved in the <exp_folder>/logs directory under the experiment directory, such as ./exp/vspw/dcfm/mitb2/logs.
Download the trained weights from here, and place the *.pth files in <exp_folder>/logs/.
python ./tools/test.py --root ./exp/vspw/dcfm/mitb2/infer.yaml --k 2The output generated during the testing process will be saved in a text file with the same name under the experiment directory, and the final testing results will be incrementally saved in results.txt under the experiment directory, such as ./exp/vspw/dcfm/mitb2/infer.txt and ./exp/vspw/dcfm/mitb2/results.txt.
The code is heavily based on the following repositories:
Thanks for their amazing works.