Application of Deep Learning-Based Methods to the Single Image Non-Uniform Blind Motion Deblurring Problem
DOI:
https://doi.org/10.51408/1963-0072Keywords:
Motion blur, Blind motion deblurring, Non-uniform blurring, Blur kernelAbstract
In this paper, we present deep learning-based blind image deblurring methods for estimating and removing a non-uniform motion blur from a single blurry image. We propose two fully convolutional neural networks (CNN) for solving the problem. The networks are trained end-to-end to reconstruct the latent sharp image directly from the given single blurry image without estimating and making any assumptions on the blur kernel, its uniformity, and noise. We demonstrate the performance of the proposed models and show that our approaches can effectively estimate and remove complex non-uniform motion blur from a single blurry image.
References
Wikipedia, (2008) Wiener Deconvolution. [Online]. Available: https://en.wikipedia.org/wiki/Wiener_deconvolution
W. Richardson, “Bayesian-based iterative method of image restoration”, Journal of the Optical Society of America, vol. 62, no. 1, pp. 55-59, 1972.
L. Lucy, “An iterative technique for the rectification of observed distributions”, The Astronomical Journal, vol. 79, no. 6, pp. 745-754, 1974.
D. Krishnan and R. Fergus, “Fast image deconvolution using hyperlaplacian priors”, Proceedings of the 23rd International Conference on Neural Information Processing Systems, Vancouver, Canada, pp. 1033–1041, 2009.
L. Rudin, S. Osher and E. Fatemi, “Nonlinear total variation based noise removal algorithms”, Physica D: Nonlinear Phenomena, vol. 60, no. 1-4, pp. 259–268, 1992.
A. Levin, Y. Weiss, F. Durand and W. Freeman, “Understanding blind deconvolution algorithms”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 12, pp. 2354–2367, 2011.
J. Pan, Z. Hu, Z. Su and M. Yang, “L0-regularized intensity and gradient prior for deblurring text images and beyond”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 2, pp. 342-355, 2017.
J. Pan, D. Sun, H. Pfister and M. Yang, “Blind image deblurring using dark channel prior”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, USA, pp. 1628-1636, 2016.
Y. Bai, G. Cheung, X. Liu and W. Gao, “Graph-Based Blind Image Deblurring From a Single Photograph”, IEEE Transactions on Image Processing, vol. 28, no. 3, pp. 1404-1418, 2019.
S. Cho and S. Lee, “Fast motion deblurring”, ACM Transactions on Graphics, vol. 28, no. 5, article 145, pp. 1-8, 2009.
S. Zheng, L. Xu and J. Jia, “Forward motion deblurring”, Proceedings of the IEEE International Conference on Computer Vision (ICCV), Sydney, Australia, pp. 1465-1472, 2013.
Wikipedia, (2016) The maximum-a-posteriori estimation. [Online]. Available: https://en.wikipedia.org/wiki/Maximum_a_posteriori_estimation
L. Xu, J. Ren, C. Liu, and J. Jia, “Deep convolutional neural network for image deconvolution”, Proceedings of the 27th International Conference on Neural Information Processing Systems, Montreal, Canada, pp. 1790–1798, 2014.
J. Zhang, J. Pan, J. Ren, et al., “Dynamic scene deblurring using spatially variant recurrent neural networks”, Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, USA, pp. 2521-2529, 2018.
T. Nimisha, V. Rengarajan and R. Ambasamudram, “Semi-Supervised Learning of Camera Motion from a Blurred Image”, Proceedings of the 25th IEEE International Conference on Image Processing (ICIP), Athens, Greece, pp. 803-807, 2018.
J. Sun, W. Cao, Z. Xu and J. Ponce, “Learning a convolutional neural network for non-uniform motion blur removal”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, USA, pp. 769-777, 2015.
S. Nah, T. Kim and K. Lee, “Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, USA, pp. 257-265, 2017.
S. Nah, (2017) The GOPRO dataset. [Online]. Available: https://seungjunnah.github.io/Datasets/gopro
K. He, X. Zhang, S. Ren and J. Sun, “Deep Residual Learning for Image Recognition”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, USA, pp. 770-778, 2016.
S. Ioffe and C. Szegedy, “Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift”, Proceedings of the 32nd International Conference on Machine Learning, Lille, France, pp. 448-456, 2015.
O. Ronneberger, P. Fischer and T. Brox, “U-Net: Convolutional Networks for Biomedical Image Segmentation”, Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), Munich, Germany, pp. 234-241, 2015.
J. Johnson, A. Alahi, and L. Fei, “Perceptual losses for real-time style transfer and super-resolution”, Proceedings of the European Conference on Computer Vision (ECCV), Amsterdam, The Netherlands, pp. 694-711, 2016.
J. Johnson, (2016) Perceptual Losses for Real-Time Style Transfer and Super-Resolution: Supplementary Material. Link for Fig. 3 a-b. [Online]. Available: https://cs.stanford.edu/people/jcjohns/papers/fast-style/fast-style-supp.pdf
Wikipedia, (2019) The mean squared error. [Online]. Available: https://en.wikipedia.org/wiki/Mean_squared_error
Wikipedia, (2017) The mean absolute error. [Online]. Available: https://en.wikipedia.org/wiki/Mean_absolute_error
Wikipedia, (2013) The peak signal-to-noise ratio. [Online]. Available: https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio
D. Kingma and J. Ba, (2017) arXiv paper page - Adam: A Method for Stochastic Optimization. [Online]. Available: https://arxiv.org/abs/1412.6980v5
Wikipedia, (2020) The stochastic gradient descent. [Online]. Available: https://en.wikipedia.org/wiki/Stochastic_gradient_descent
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
