Direction-coded Temporal U-shape Module for Multiframe Infrared Small Target Detection

Pytorch implementation of our Direction-coded Temporal U-shape Module (DTUM). [Paper]

Contributions

• We design a direction-coded convolution block (DCCB) to extract motion information by encoding the motion direction into features.
• Based on DCCB, we propose a simple yet effective DTUM to enhance dim targets while suppressing false alarms for MIRST detection. DTUM can be equipped with most single-frame networks to leverage spatial–temporal information for MIRST.
• We develop an NUDT-MIRSDT dataset with both mask and point-level annotations for MIRST detection.
• We propose a hard point mining (HPM) loss function, which works wonders in point-supervised learning!

Requirements

• Python 3
• torch
• mmdet
• tqdm
• DCNv2
• scikit-image
  
  

Datasets

NUDT-MIRSDT   [Baidu download dir] (Extraction code: 5whn) is a synthesized dataset, which contains 120 sequences. We use 80 sequences for training and 20 sequences for test. We divide the test set into two subsets according to their SNR ((0, 3], (3, 10)).

In the test set, targets in 8 sequences are so weak (SNR lower than 3). It is very challenging to detect these targets. The test set includes Sequence[47, 56, 59, 76, 92, 101, 105, 119].

Train on NUDT-MIRSDT Dataset

python train.py --model 'ResUNet_DTUM' --loss_func 'fullySup' --train 1 --test 0 --fullySupervised True
python train.py --model 'DNANet_DTUM' --loss_func 'fullySup1' --train 1 --test 0 --fullySupervised True --SpatialDeepSup False

Test on NUDT-MIRSDT Dataset

python train.py --model 'ResUNet_DTUM' --loss_func 'fullySup' --train 0 --test 1 --pth_path [trained model path]

Train on IRDST Dataset (or other datasets)

python train.py --dataset IRDST --align --model 'ResUNet_DTUM' --loss_func 'fullySup' --train 1 --test 0 --fullySupervised True

If the background in a sequence is not aligned, the parameter of "align" needs to be set to True.

Results and Trained Models

Quantative Results

on NUDT-MIRSDT (SNR≤3)

Model	Pd (x10(-2))	Fa (x10(-5))	AUC
ALCNet +DTUM	56.144	0.931	0.9489
Res-UNet+DTUM	91.682	2.369	0.9921	[Weights]
DNANet +DTUM	85.444	1.118	0.9882
ISNet +DTUM	50.662	0.646	0.9482
UIUNet +DTUM	72.023	1.916	0.9933

on NUDT-MIRSDT (SNR＞3)

Model	Pd (x10(-2))	Fa (x10(-5))	AUC
ALCNet +DTUM	99.500	2.370	0.9988
Res-UNet+DTUM	100	3.415	0.9988	[Weights]
DNANet +DTUM	99.833	3.633	0.9988
ISNet +DTUM	99.750	3.448	0.9988
UIUNet +DTUM	99.833	3.578	0.9988

on NUDT-MIRSDT (all)

Model	Pd (x10(-2))	Fa (x10(-5))	AUC
ALCNet +DTUM	86.235	1.798	0.9818
Res-UNet+DTUM	97.455	2.999	0.9964	[Weights]
DNANet +DTUM	95.431	2.620	0.9951
ISNet +DTUM	84.731	2.334	0.9816
UIUNet +DTUM	91.324	2.917	0.9972

Citiation

@article{li2023direction,
  title={Direction-Coded Temporal U-Shape Module for Multiframe Infrared Small Target Detection.},
  journal={IEEE Transactions on Neural Networks and Learning Systems},
  year={2023}
}

Contact

Welcome to raise issues or email to liruojing@nudt.edu.cn for any question.