A Multispectral Decomposition and Frequency-Based Framework for Salient Object Detection in Remote Sensing Images
DOI:
https://doi.org/10.51408/1963-0124Keywords:
Saliency map, Object detection, Multispectral decomposition, Band selection, Remote sensing, EntropyAbstract
Salient object detection (SOD) aims to identify the most visually prominent objects in images, crucial for tasks like image segmentation, visual tracking, autonomous navigation, and photo cropping. While SOD has been extensively studied in natural scene RGB images, detecting salient objects in remote sensing images remains underexplored due to varying spatial resolutions and complex scenes.
This paper presents a novel framework for SOD called Multispectral Decomposition Network (MSD-Net) in remote sensing 3-band RGB images, combining Multispectral Decomposition and Frequency-based Saliency detection. The framework includes three key steps: (i) Multispectral Decomposition: Decomposing a 3-band RGB image into 32 multispectral bands to enhance feature capture across spectral domains; (ii) Synthetic RGB Reconstruction: Using a new entropy-based measure to select the most informative bands in salient regions by analyzing frequency domain and constructing synthetic RGB image; and (iii) Saliency Fusion and Object Detection: training a segmentation network on the fusion of synthetic RGB image and input image for improved accuracy. Comprehensive evaluations of public datasets demonstrate that the proposed method performs better than state-of-the-art (SOTA) models and offers a robust solution for detecting salient objects in complex remote sensing images by integrating multispectral and frequency-based techniques.
References
L. Zhang and Li. Zhang, “Artificial intelligence for remote sensing data analysis: A review of challenges and opportunities”, IEEE Geoscience and Remote Sensing Magazine, vol. 10, no. 2, pp. 270-294, 2002.
X. Wang et al., “Salient object detection: a mini review”, Frontiers in Signal Processing, vol. 4, 1356793, 2024.
W. Wang et al., “Salient object detection in the deep learning era: An in-depth survey”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 6, pp. 3239-3259, 2021.
A. Borji et al., “Salient object detection: A benchmark”, IEEE Transactions on Image Processing, vol. 24, no. 12, pp. 5706-5722, 2015.
X. Hou and L. Zhang, “Saliency detection: A spectral residual approach”, IEEE Conference on Computer Vision and Pattern Recognition, 2007, DOI: 10.1109/CVPR.2007.383267
Cheng Ming-Ming, et al., “Global contrast based salient region detection”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, no. 3, pp. 569-582, 2014.
Li. Guanbin and Yu. Yizhou, “Visual saliency based on multiscale deep features”, Proceedings of the IEEE Conference on Computer Vision And Pattern Recognition. 2015.
J. Long, E. Shelhamer and T. Darrell, “Fully convolutional networks for semantic segmentation”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, DOI: 10.1109/CVPR.2015.7298965
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition”, arXiv preprint arXiv:1409.1556 (2014).
K. He et al., “Deep residual learning for image recognition”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, DOI: 10.1109/CVPR.2016.90
A. Howard, “Mobilenets: Efficient convolutional neural networks for mobile vision applications”, arXiv preprint arXiv:1704.04861, 2017.
G. Fang et al., “Video saliency detection using object proposals”, IEEE Transactions on Cybernetics, vol. 48, no.11, pp. 3159-3170, 2017.
A. Dosovitskiy, “An image is worth 16x16 words: Transformers for image recognition at scale”, arXiv preprint arXiv:2010.11929, 2020.
N. Liu et al., “Visual saliency transformer”, Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021.
Q. Zhang et al., “Dense attention fluid network for salient object detection in optical remote sensing images”, IEEE Transactions on Image Processing, vol. 30, pp. 1305-1317, 2020.
S. C. Nercessian, K. A. Panetta and S. S. Agaian, “Non-Linear Direct Multi-Scale Image Enhancement Based on the Luminance and Contrast Masking Characteristics of the Human Visual System”, IEEE Transactions on Image Processing, vol. 22, no. 9, pp. 3549-3561, 2013.
J. Xia, K. Panetta and S. Agaian, “Wavelet transform coefficient histogram-based image enhancement algorithms”, Proc. SPIE 7708, 770812, 2010.
A. Grigoryan, J. Jenkinson and S. Agaian, “Quaternion Fourier transform based alpha-rooting method for color image measurement and enhancement”, Signal Processing, vol. 109, pp. 269-289, 2015.
E. Wharton, K. Panetta and S. Agaian, “Human visual system based multi-histogram equalization for non-uniform illumination and shadow correction”, Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, vol. 1, pp. I-729–I-732, 2007.
S. Nercessian, S. Agaian and K. Panetta, “An image similarity measure using enhanced human visual system characteristics”, Proceedings of the SPIE, Mobile Multimedia/Image Processing, Security, and Applications, vol. 8063, p. 806310, 2011.
S. Agaian, “Visual morphology”, Proc. IS&T/SPIE's Symposium on Electronic Imaging Science & Technology, vol. 3304, pp. 153–163, 1999.
R. Kogan, S. Agaian and K. Panetta, “Visualization using rational morphology and zonal magnitude reduction”, IX Proceedings of IS&T/SPIE's Symposium on Electronic Imaging Science & Technology, San Jose, CA, vol. 3304, pp. 153–163, 1998.
H. D. Cheng, Y.-H. Chen and Y. Sun, “A novel fuzzy entropy approach to image enhancement and thresholding”, Signal Process, vol. 75, pp. 277–301, 1999.
S. Agaian, B. Silver and K. Panetta, “Transform coefficient histogram-based image enhancement algorithms using contrast entropy”, IEEE Trans. Image Process., vol. 16, pp. 741–758, 2007.
A. M. Reza, “Realization of the contrast limited adaptive histogram equalization (CLAHE) for real-time image enhancement”, Journal of VLSI Signal Processing Systems for Signal, Image and Video Technology, vol. 38, pp. 35-44, 2004.
T. Trongtirakul, S. Agaian and S. Wu, “Adaptive Single Low-Light Image Enhancement by Fractional Stretching in Logarithmic Domain”, IEEE Access, vol. 11, pp. 143936-143947, 2023.
B. Arad et al., “Ntire 2022 spectral recovery challenge and data set”, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
Y. Cai et al., “Mst++: Multi-stage spectral-wise transformer for efficient spectral reconstruction”, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
H. Gasparyan, T. Davtyan and S. Agaian, “A novel framework for solar panel segmentation from remote sensing images: Utilizing Chebyshev transformer and hyperspectral decomposition”, IEEE Transactions on Geoscience and Remote Sensing, vol. 62, 2024, DOI: 10.1109/TGRS.2024.3386402.
Li.-Ch. Chen, “Rethinking atrous convolution for semantic image segmentation”, arXiv preprint arXiv:1706.05587, 2017.
Ch. Li et al., “Nested network with two-stream pyramid for salient object detection in optical remote sensing images”, IEEE Transactions on Geoscience and Remote Sensing, vol. 57, no. 11, pp. 9156-9166, 2019.
G. Cheng, J. Han and Xi. Lu, “Remote sensing image scene classification: Benchmark and state of the art”, Proceedings of the IEEE 105.10, pp. 1865-1883, 2017.
H. Jiang et al., “Multi-resolution dataset for photovoltaic panel segmentation from satellite and aerial imagery”, Earth System Science Data 13.11, pp. 5389-5401, 2021.
D.-P. Fan et al., “Structure-measure: A new way to evaluate foreground maps”, Proceedings of the IEEE International Conference on Computer Vision, 2017. DOI: 10.1109/ICCV.2017.487
D.-P. Fan et al., “Enhanced-alignment measure for binary foreground map evaluation”, arXiv preprint arXiv:1805.10421, 2018.
R. Achanta et al., “Frequency-tuned salient region detection”, IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2009.
B. Silver, S. Agaian and K. Panetta, “Contrast entropy based image enhancement and logarithmic transform coefficient histogram shifting”, Proceedings of IEEE ICASSP, pp. 633–636, 2005.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Hayk. A. Gasparyan
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
