Yuhang Gan, Wenjie Xuan, Zhiming Luo, Lei Fang, Zengmao Wang, Juhua Liu, Bo Du
This is the PyTorch implementation for Semantic-Aware Change Detection network, namely SA-CDNet, which transfers the common knowledge of the visual foundation models to change detection. Moreover, a single-temporal semantic pre-training strategy is proposed to enrich the semantic prior of the network.
[Under Review]
- [2025/01/13]: Codes for training and evaluation are released. This repo is an updated version of SA-CD by the co-author Zhiming Luo.
- Release training and inference codes. Instructions on dataset preparation and checkpoints are also provided.
- A semantic-aware change detection network, namely SA-CDNet. We propose an advanced SA-CDNet for change detection, which inherits the rich knowledge of natural images from the vision foundation model and adapts these semantic priors to remote sensing change detection. Additionally, a dual-stream feature decoder is derived to mimic the human visual paradigm, which integrates semantic-aware and difference-aware features for robust predictions.
- A well-designed single-temporal semantic pre-training strategy. To relieve the scarcity of annotated change detection data, we construct pseudo-change detection data from remote sensing semantic segmentation datasets to pre-train the network. Based on our SA-CDNet, an extra segmentation branch is introduced for improved semantic understanding through discriminating landscapes in single-temporal images.
- **Extensive experiments on five challenging benchmarks. ** We conducted experiments on five benchmarks including LEVIR-CD, LEVIR-CD+, S2Looking, WHU-CD for building changes, and the WHU Cultivated Land for farmland changes. Our method achieves SOTA performance on all datasets. Comprehensive ablation studies proved the effectiveness of our model and pre-training strategy.
-
The architecture of our proposed SA-CDNet, which comprises of a vision foundation model-based feature encoder with change detection adapter, and a dual-stream feature decoder to combine semantic-aware and difference-aware features for robust change detection.
-
Our single-temporal semantic pre-training strategy, where we construct pseudo-change detection data from remote sensing semantic segmentation datasets as the pre-training dataset, and an additional single-temporal semantic segmentation task is introduced into the pre-training to improve the semantic understanding of bi-temporal images.
Recommended: Python=3.7 torch=1.9.0 CUDA=11.7
# set up repository
git https://github.com/DREAMXFAR/SA-CDNet.git
cd SA-CDNet-master
# install conda environment
conda create -n sacdnet python=3.7
conda activate sacdnet
pip install -r requirements.txtYou can download the following checkpoints and put them in checkpoints/ and fastsam_model/.
| Model | Baidu Yun | Notations |
|---|---|---|
| FastSAM | link | Pre-trained FastSAM models. |
| SA-CDNet (pre-trained) | link | Our SA-CDNet pre-trained on pseudo-change data. |
| SA-CDNet (fine-tuned) | link | Our final SA-CDNet after fine-tuning on change detection data. |
Here we suppose you are in
SA-CDNet-master/
-
The whole project is supposed to organized as follows.
# directory SA-CD ├── checkpoints # model checkpoints, need to download model checkpoints │ ├── SANet │ │ ├── LEVIR │ │ │ ├── no_pretrain.pth │ │ │ └── with_pretrain.pth │ │ ├── LEVIR+ │ │ │ ├── no_pretrain.pth │ │ │ └── with_pretrain.pth │ │ ├── S2Looking │ │ │ ├── no_pretrain.pth │ │ │ └── with_pretrain.pth │ │ ├── WHU │ │ │ ├── no_pretrain.pth │ │ │ └── with_pretrain.pth │ │ └── WHU_CUL │ │ ├── no_pretrain.pth │ │ └── with_pretrain.pth │ └── pre_train # pre-trained models │ ├── Building │ │ └── pretrain_Building.pth │ └── Cultivalted land │ └── pretrain_cultivalted.pth ├── datasets # data loader │ ├── Levir_CD.py │ ├── Levir_CD2.py │ ├── data_utils.py │ ├── data_utils2.py │ ├── pre_CD.py │ ├── pre_CD2.py │ └── pre_CD3.py ├── fastsam_model # checkpoints of vision foundation models, also need to download │ ├── FastSAM-s.pt │ └── FastSAM-x.pt ├── models # model definitions │ ├── FastSAM │ ├── SAM_Fusion2.py │ └── preSAM_SNUNet4.py ├── pretrain.py ├── train.py ├── eval.py ├── ultralytics └── utils
-
Download the datasets first. Here we release all the datasets we used for pre-training and fine-tuning. Specifically, the single temporal semantic segmentation datasets are used to construct pseudo-change data for pre-training and change detection datasets are used for fine-tuning.
Dataset Baidu Yun Dataset Category LEVIRlink change detection LEVIR+link change detection S2Lookinglink change detection WHU-CDlink change detection WHU Cultivate Land Datasetlink change detection WHU-Buildinglink remote-sensing semantic segmentation INRIA-Buildinglink remote-sensing semantic segmentation LoveDAlink remote-sensing semantic segmentation DLCCClink remote-sensing semantic segmentation AIRSlink remote-sensing semantic segmentation
-
Configure the dataset path in the following files.
# for single dataset, in ./datasets/pre_CD.py, line 16 root = "path/to/pretraining/dataset" # for two datasets, in ./datasets/pre_CD2.py, line 16, 17 root1 = "/path/to/the/first/dataset" root2 = "/path/to/the/second/dataset" # for three datasets, in ./datasets/pre_CD3.py, line 16 root1 = "/path/to/first/dataset" root2 = "/path/to/second/dataset" root3 = "/path/to/third/dataset"
-
Update the model path of FastSAM in the following files.
# ./models/preSAM_SUNNet4.py, line 235 model_name: str = '/path/to/fastsam_model/FastSAM-x.pt' # ./models/SAM_Fusion2.py, line 160 model_name: str = '/path/to/fastsam_model/FastSAM-x.pt'
-
Update the directory path of save models in the following file.
# ./pretrain.py, line 22 and 26 NET_NAME = '/path/of/model/name' DATA_NAME = '/path/of/data/name' # more detailed pre-training settings like lr are listed in line 29.
-
Update the path of datasets in the following file.
# ./datasets/Levir_CD.py, line 16 root = '/path/of/dataset' # data path
-
Configure the pathes and parameters for training.
# the parameter settings for training, in ./train.py # the path of saved models, in line 20 and 24 NET_NAME = '/path/of/model/name' DATA_NAME = '/path/of/data/name' # enable to load pre-trained models, line 40 'load_premodel': True # the path of pre-trained models, line 43 'chkpt_path': '/path/to/pretrain/model', # pretrain model path # more detailed training settings like lr are listed in line 26.
-
Configure the validation settings.
# the parameter settings for validation, in ./eval.py # the model save path, in line 19 and 22 NET_NAME = '/path/of/model/name' DATA_NAME = '/path/of/data/name' # the model name for validation, in line 38, 42 'chkpt_path': '/path/to/ckpt', 'load_path': '/path/to/load/data',
-
To inference, you can use the validation codes.
-
Quantitative results. The performance of the proposed method is evaluated with and without pretraining on multiple public datasets. Results are presented in terms of Precision (P), Recall (R), and F1-Score (F1).
Datasets Methods P(%) R(%) F1(%) LEVIR SA - CD w/o pretrain 92.69 89.66 91.15 SA - CD w/ pretrain 91.77 91.28 91.53 LEVIR + SA - CD w/o pretrain 86.88 79.15 82.83 SA - CD w/ pretrain 85.55 83.44 84.43 S2Looking SA - CD w/o pretrain 81.25 54.73 65.40 SA - CD w/ pretrain 81.28 56.24 66.48 WHU SA - CD w/o pretrain 94.66 91.22 92.91 SA - CD w/ pretrain 95.29 93.67 94.47 WHU Cultivate Land SA - CD w/o pretrain 72.65 73.93 73.28 SA - CD w/ pretrain 77.74 72.82 75.20 -
Qualitative results. Visualized comparisons with existing SOTA methods on five benchmark change detection datasets.
- None
- Our implementation is greatly based on the FastSAM, SAM-CD, Thanks for their wonderful works and all contributors.
If you find our findings helpful in your research, please consider giving this repository a ⭐ and citing:
@misc{gan2024detectchangeslikehumans,
title={Detect Changes like Humans: Incorporating Semantic Priors for Improved Change Detection},
author={Yuhang Gan and Wenjie Xuan and Zhiming Luo and Lei Fang and Zengmao Wang and Juhua Liu and Bo Du},
year={2024},
eprint={2412.16918},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2412.16918},
}