Enhancing Thermal Image Classification with Novel Quality Metric-Based Augmentation Techniques
DOI:
https://doi.org/10.51408/1963-0125Keywords:
Dataset augmentation, Image quality assessment, Thermal image classificationAbstract
Thermal image classification is critical in various applications, particularly fault detection and monitoring systems such as photovoltaic (PV) modules. However, a common challenge in these fields is the limited availability of large-scale, labeled thermal image datasets. To address this, color image augmentation is widely adopted in machine learning to artificially increase the size and diversity of training datasets, improving model generalization. Traditional augmentation techniques, such as geometric transformations, provide some benefits but may fail to fully capture the unique characteristics inherent in thermal images, which often have lower contrast and different noise patterns than visible spectrum images. So, we argued we need to develop a novel augmentation technique for thermal imaging, where data collection is costly and time-consuming.
Our research proposes a novel offline augmentation technique guided by quality metrics to enhance the performance of thermal image binary classification models. By leveraging domain-specific quality metrics, such as image clarity, thermal contrast, and noise levels, we optimize the oversampling process for thermal datasets. For example, starting with a dataset of x images, we generate y additional thermal images, resulting in a total of x + y images used to train the deep learning classification framework. Using a dataset of PV module defects, we demonstrate the effectiveness of our quality metric-based oversampling strategy across several state-of-the-art image classification networks. Our approach outperforms traditional augmentation methods regarding classification accuracy and robustness, including geometric transformations and standard image enhancement techniques. The practical implications of our research are significant, as it provides a more effective and efficient way to improve model performance for thermal imaging tasks, mainly when data availability is limited.
References
M. W. Akram, G. Li, Y. Jin, X. Chen, C. Zhu, X. Zhao, M. Aleem, and A. Ahmad, “Improved outdoor thermography and processing of infrared images for defect detection in pv modules,” Solar Energy, vol. 190, pp. 549–560, 2019.
R. H. F. Alves, G. A. de Deus Junior, E. G. Marra, and R. P. Lemos, “Automatic fault classification in photovoltaic modules using convolutional neural networks,” Renewable Energy, vol. 179, pp. 502–516, 2021.
T. Trongtirakul, S. Agaian, and A. Oulefki, “Automated tumor segmentation in thermographic breast images,” Mathematical Biosciences and Engineering, vol. 20, no. 9, pp. 16786–16806, 2023.
M. d. F. O. Baffa and L. G. Lattari, “Convolutional neural networks for static and dynamic breast infrared imaging classification,” in 2018 31st SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pp. 174–181, IEEE, 2018.
D. Korkmaz and H. Acikgoz, “An efficient fault classification method in solar photovoltaic modules using transfer learning and multi-scale convolutional neural network,” Engineering Applications of Artificial Intelligence, vol. 113, p. 104959, 2022.
Z. Yang, R. O. Sinnott, J. Bailey, and Q. Ke, “A survey of automated data augmentation algorithms for deep learning-based image classification tasks,” Knowledge and Information Systems, vol. 65, no. 7, pp. 2805–2861, 2023.
W. Li, C. Chen, M. Zhang, H. Li, and Q. Du, “Data augmentation for hyperspectral image classification with deep cnn,” IEEE Geoscience and Remote Sensing Letters, vol. 16, no. 4, pp. 593–597, 2018.
R. F. Pamungkas, I. B. K. Y. Utama, and Y. M. Jang, “A novel approach for efficient solar panel fault classification using coupled udensenet,” Sensors, vol. 23, no. 10, p. 4918, 2023.
C. Tang, H. Ren, J. Xia, F. Wang, and J. Lu, “Automatic defect identification of pv panels with ir images through unmanned aircraft,” IET Renewable Power Generation, vol. 17, no. 12, pp. 3108–3119, 2023.
M. Millendorf, E. Obropta, and N. Vadhavkar, “Infrared solar module dataset for anomaly detection,” in Proc. Int. Conf. Learn. Represent, 2020.
A. Krizhevsky, “One weird trick for parallelizing convolutional neural networks,” arXiv preprint arXiv:1404.5997, 2014.
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778, 2016.
Z. Liu, H. Hu, Y. Lin, Z. Yao, Z. Xie, Y. Wei, J. Ning, Y. Cao, Z. Zhang, L. Dong, et al., “Swin transformer v2: Scaling up capacity and resolution,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 12009–12019, 2022.
F. N. Iandola, “Squeezenet: Alexnet-level accuracy with 50x fewer parameters and¡ 0.5 mb model size,” arXiv preprint arXiv:1602.07360, 2016.
N. Ma, X. Zhang, H.-T. Zheng, and J. Sun, “Shufflenet v2: Practical guidelines for efficient cnn architecture design,” in Proceedings of the European conference on computer vision (ECCV), pp. 116–131, 2018.
A. Howard, M. Sandler, G. Chu, L.-C. Chen, B. Chen, M. Tan, W. Wang, Y. Zhu, R. Pang, V. Vasudevan, et al., “Searching for mobilenetv3,” in Proceedings of the IEEE/CVF international conference on computer vision, pp. 1314–1324, 2019.
H. Ayunts, A. Grigoryan, and S. Agaian, “Novel entropy for enhanced thermal imaging and uncertainty quantification,” Entropy, vol. 26, no. 5, p. 374, 2024.
K. Panetta, C. Gao, and S. Agaian, “No reference color image contrast and quality measures,” IEEE transactions on Consumer Electronics, vol. 59, no. 3, pp. 643–651, 2013.
H. Ayunts and S. Agaian, “No-reference quality metrics for image decolorization,” IEEE Transactions on Consumer Electronics, vol. 69, no. 4, pp. 1177–1185, 2023.
A. Yunts and S. Agaian, "No-reference quality metrics for image decolorization," IEEE Transactions on Consumer Electronics, vol. 69, no. 4, pp. 1177–1185, 2023.
T. Trongtirakul and S. Agaian, "Unsupervised and optimized thermal image quality enhancement and visual surveillance applications," Signal Processing: Image Communication, vol. 105, pp. 1167–114, 2022.
P. Bromiley, N. Thacker, and E. Bouhova-Thacker, "Shannon entropy, renyi entropy, and information," Statistics and Inf. Series (2004-004), vol. 9, no. 2004, pp. 2–8, 2004.
S. S. Agaian, B. Silver, and K. A. Panetta, "Transform coefficient histogram-based image enhancement algorithms using contrast entropy," IEEE Transactions on Image Processing, vol. 16, no. 3, pp. 741–758, 2007.
A. M. Grigoryan and S. S. Agaian, "Image processing contrast enhancement," Wiley Encyclopedia of Electrical and Electronics Engineering, pp. 1–22, 1999.
A. M. Grigoryan, A. John, and S. S. Agaian, "Color image enhancement of medical images using alpha-rooting and zonal alpha-rooting methods on 2D qdft," in Medical Imaging 2017: Image Perception, Observer Performance, and Technology Assessment, vol. 10136, pp. 325–333, SPIE, 2017.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Hrach Y. Ayunts
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
