Abstract
In recent years, deep learning emerges as one promising technique for solving many ill-posed inverse problems in image recovery, and most deep-learning-based solutions are based on supervised learning. Motivated by the practical value of reducing the cost and complexity of constructing labeled training datasets, this paper proposed a self-supervised deep learning approach for image recovery, which is dataset-free. Built upon Bayesian deep network, the proposed method trains a network with random weights that predicts the target image for recovery with uncertainty. Such uncertainty enables the prediction of the target image with small mean squared error by averaging multiple predictions. The proposed method is applied for image reconstruction in compressive sensing (CS), i.e., reconstructing an image from few measurements. The experiments showed that the proposed dataset-free deep learning method not only significantly outperforms traditional non-learning methods, but also is very competitive to the state-of-the-art supervised deep learning methods, especially when the measurements are few and noisy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Arce, G., Brady, D., Carin, L., Arguello, H., Kittle, D.: Compressive coded aperture spectral imaging: an introduction. IEEE Signal Process. Mag. 31(1), 105–115 (2013)
Baldi, P., Sadowski, P.J.: Understanding dropout. In: NeurIPS, pp. 2814–2822 (2013)
Barber, D., Bishop, C.M.: Ensemble learning in Bayesian neural networks. Nato ASI Ser. F Comput. Syst. Sci. 168, 215–238 (1998)
Bernstein, M.A., Fain, S.B., Riederer, S.J.: Effect of windowing and zero-filled reconstruction of MRI data on spatial resolution and acquisition strategy. J. Magn. Reson. Imaging 14(3), 270–280 (2001)
Blundell, C., Cornebise, J., Kavukcuoglu, K., Wierstra, D.: Weight uncertainty in neural network. In: ICML, pp. 1613–1622 (2015)
Cai, J., Ji, H., Liu, C., Shen, Z.: Blind motion deblurring from a single image using sparse approximation. In: CVPR, pp. 104–111 (2009)
Candes, E.J., Tao, T.: Near-optimal signal recovery from random projections: universal encoding strategies? IEEE Trans. Inf. Theor. 52(12), 5406–5425 (2006)
Chen, G., Tang, J., Leng, S.: Prior image constrained compressed sensing (PICCS): a method to accurately reconstruct dynamic CT images from highly undersampled projection data sets. Med. Phys. 35(2), 660–663 (2008)
Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3-d transform-domain collaborative filtering. IEEE Trans. Image Process. 16(8), 2080–2095 (2007)
Danielyan, A., Katkovnik, V., Egiazarian, K.: Bm3d frames and variational image deblurring. IEEE Trans. Image Process. 21(4), 1715–1728 (2011)
Ding, Q., Chen, G., Zhang, X., Huang, Q., Ji, H., Gao, H.: Low-dose CT with deep learning regularization via proximal forward backward splitting. Phys. Med. Biol. 65(12), 125009 (2020)
Dong, W., Shi, G., Li, X., Ma, Y., Huang, F.: Compressive sensing via nonlocal low-rank regularization. IEEE Trans. Image Process. 23(8), 3618–3632 (2014)
Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)
Duarte, M., et al.: Single-pixel imaging via compressive sampling. IEEE Signal Process. Mag. 25(2), 83–91 (2008)
Gal, Y., Ghahramani, Z.: Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In: ICML, pp. 1050–1059 (2016)
Gamper, U., Boesiger, P., Kozerke, S.: Compressed sensing in dynamic MRI. Magn. Reson. Med. 59(2), 365–373 (2008)
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016)
He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: ICCV, pp. 1026–1034 (2015)
Heckel, R., Hand, P.: Deep decoder: Concise image representations from untrained non-convolutional networks. arXiv preprint arXiv:1810.03982 (2018)
Kendall, A., Gal, Y.: What uncertainties do we need in Bayesian deep learning for computer vision? In: NeurIPS, pp. 5574–5584 (2017)
Kulkarni, K., Lohit, S., Turaga, P., Kerviche, R., Ashok, A.: Reconnet: Non-iterative reconstruction of images from compressively sensed measurements. In: CVPR, pp. 449–458 (2016)
Lakshminarayanan, B., Pritzel, A., Blundell, C.: Simple and scalable predictive uncertainty estimation using deep ensembles. In: NeurIPS, pp. 6402–6413 (2017)
Li, C., Yin, W., Jiang, H., Zhang, Y.: An efficient augmented lagrangian method with applications to total variation minimization. Comput. Optim. Appl. 56(3), 507–530 (2013)
Li, M., Fan, Z., Ji, H., Shen, Z.: Wavelet frame based algorithm for 3d reconstruction in electron microscopy. SIAM J. Sci. Comput. 36(1), B45–B69 (2014)
Liu, J., Chen, N., Ji, H.: Learnable Douglas-Rachford iteration and its applications in dot imaging. Inverse Prob. Imaging 14(4), 683 (2020)
Liu, J., Kuang, T., Zhang, X.: Image reconstruction by splitting deep learning regularization from iterative inversion. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11070, pp. 224–231. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00928-1_26
Lustig, M., Donoho, D., Pauly, J.M.: Sparse MRI: The application of compressed sensing for rapid MR imaging. Magn. Reson. Med. 58(6), 1182–1195 (2007)
Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: ICCV, vol. 2, pp. 416–423. IEEE (2001)
Meinhardt, T., Moller, M., Hazirbas, C., Cremers, D.: Learning proximal operators: Using denoising networks for regularizing inverse imaging problems. In: ICCV, pp. 1781–1790 (2017)
Metzler, C.A., Maleki, A., Baraniuk, R.: Bm3d-amp: a new image recovery algorithm based on bm3d denoising. In: ICIP, pp. 3116–3120. IEEE (2015)
Metzler, C.A., Maleki, A., Baraniuk, R.: From denoising to compressed sensing. IEEE Trans. Inf. Theory 62(9), 5117–5144 (2016)
Mousavi, A., Patel, A., Baraniuk, R.: A deep learning approach to structured signal recovery. In: Allerton, pp. 1336–1343. IEEE (2015)
Nan, Y., Quan, Y., Ji, H.: Variational-EM-based deep learning for noise-blind image deblurring. In: CVPR, pp. 3626–3635 (June 2020)
Nan, Y., Ji, H.: Deep learning for handling kernel/model uncertainty in image deconvolution. In: CVPR, pp. 2388–2397 (June 2020)
Quan, Y., Chen, M., Pang, T., Ji, H.: Self2self with dropout: Learning self-supervised denoising from single image. In: CVPR, pp. 1890–1898 (2020)
Quan, Y., Ji, H., Shen, Z.: Data-driven multi-scale non-local wavelet frame construction and image recovery. J. Sci. Comput. 63(2), 307–329 (2015). https://doi.org/10.1007/s10915-014-9893-2
Schuler, C., B., C., Harmeling, S., Scholkopf, B.: A machine learning approach for non-blind image deconvolution. In: CVPR, pp. 1067–1074 (2013)
Shi, W., Jiang, F., Liu, S., Zhao, D.: Scalable convolutional neural network for image compressed sensing. In: CVPR, pp. 12290–12299 (2019)
Soltanayev, S., Chun, S.: Training deep learning based denoisers without ground truth data. In: NeurIPS, pp. 3257–3267 (2018)
Tang, J., Nett, B.E., Chen, G.: Performance comparison between total variation (TV)-based compressed sensing and statistical iterative reconstruction algorithms. Phys. Med. Biol. 54(19), 5781 (2009)
Ulyanov, D., Vedaldi, A., Lempitsky, V.: Deep image prior. In: CVPR, pp. 9446–9454 (2018)
Xu, K., Zhang, Z., Ren, F.: Lapran: A scalable laplacian pyramid reconstructive adversarial network for flexible compressive sensing reconstruction. In: ECCV, pp. 485–500 (2018)
Xu, L., Ren, J.S., Liu, C., Jia, J.: Deep convolutional neural network for image deconvolution. In: NIPS, pp. 1790–1798 (2014)
Yang, Y., Sun, J., Li, H., Xu, Z.: Deep ADMM-Net for compressive sensing MRI. In: NeurIPS, pp. 10–18 (2016)
Zhang, J., Ghanem, B.: Ista-net: Interpretable optimization-inspired deep network for image compressive sensing. In: CVPR, pp. 1828–1837 (2018)
Zhang, J., Pan, J., Lai, W.S., Lau, R.W., Yang, M.H.: Learning fully convolutional networks for iterative non-blind deconvolution. In: CVPR, pp. 3817–3825 (2017)
Zhussip, M., Soltanayev, S., Chun, S.: Training deep learning based image denoisers from undersampled measurements without ground truth and without image prior. In: CVPR, pp. 10255–10264 (2019)
Acknowledgment
Tongyao Pang and Hui Ji would like to acknowledge the support from Singapore MOE Academic Research Fund (AcRF) Tier 2 research project (MOE2017-T2-2-156), and Yuhui Quan would like to acknowledge the support of National Natural Science Foundation of China (61872151, U1611461).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Pang, T., Quan, Y., Ji, H. (2020). Self-supervised Bayesian Deep Learning for Image Recovery with Applications to Compressive Sensing. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, JM. (eds) Computer Vision – ECCV 2020. ECCV 2020. Lecture Notes in Computer Science(), vol 12356. Springer, Cham. https://doi.org/10.1007/978-3-030-58621-8_28
Download citation
DOI: https://doi.org/10.1007/978-3-030-58621-8_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-58620-1
Online ISBN: 978-3-030-58621-8
eBook Packages: Computer ScienceComputer Science (R0)