Infrared and Visible Image Fusion Method Based on Learnable Joint Sparse Low-Rank Decomposition

Abstract

Deep learning-based separation representation methods have achieved promising performance in image fusion tasks. This is attributed to the fact that the network architecture plays a very important role in the decoupling process. However, building a superior fusion network based on decoupling tasks is difficult in general. Most of the current network constructions for separated representations are still black-boxed design. To overcome this problem, it is proposed to implement decoupled task based on traditional optimization decomposition method and establish a link between the optimal solution and network architecture to construct interpretable separated representation network. In this paper, we propose a deep learning-based learnable joint sparse low-rank separation representation method for multi-source image fusion (LJSLRnet). It avoids blind empirical network design by transforming the optimization problem. Specifically, we propose a learn-able joint sparse low rank model (LJSLR) and construct a LJSLR-based decou-pling module. The LJSLR model is designed as an iterative optimization problem based on matrix multiplication, and then the learned convolutional sparse coding (LCSC) is utilized to convert matrix multiplication into convolutional operation, which is the core of constructing LJSLR-based decoupling module. Next, the it-erative process of optimization is replaced by feedforward network based on iterative threshold shrinkage. To enhance the decoupling ability and robustness of the network, the low-rank sparse loss function and cross-reconstruction loss function are developed. Compared with the existing methods, the experiments verify the effectiveness of the proposed method.