A multi-color and multistage collaborative network guided by refined transmission prior for underwater image enhancement

Abstract

Due to the attenuation and scattering properties of light in underwater scenes, underwater images are generally subject to color deviations and low contrast, which is not conducive to the follow-up algorithms. To alleviate these two problems, we propose a multi-color and multistage collaborative network guided by refined transmission, called MMCGT, to accomplish the enhancement tasks. Specifically, we first design an accurate method of parameter estimation to derive transmission priors that are more suitable for underwater imaging, such as min–max conversion, low-pass filter-based estimation and saturation detection. Then, we propose a multistage and multi-color space collaborative network to decompose the underwater image enhancement task into more straightforward and controllable subtasks, including colorful feature extraction, color deviation detection, and image position information retention. Finally, we apply the derived transmission prior to the transmission-guided block of the network and effectively combine the well-designed physical-inconsistency loss with Charbonnier loss and VGG loss to guide the MMCGT to compensate for the quality-degraded regions better. Extensive experiments show that MMCGT achieves better evaluation results under the dual guidance of physics and deep learning than the competing methods in visual quality and quantitative metrics.

Appendix

According to the definition of the ambient light(Eq. (9)) and the underwater image(Eq. (8)) in the above text, we can easily get the value range of $t\left( \lambda \right) _{\text {rough}}^{d\left( x \right) }$ as follows:

$$\begin{aligned} \begin{aligned} 0 \le \max \begin{pmatrix} \displaystyle \max _{y\in \Omega \left( x \right) }\frac{I_{r}\left( y \right) -B_{r}\left( y \right) }{1-B_{r}\left( y \right) }, \\ \displaystyle \max _{y\in \Omega \left( x \right) }\frac{B_{g}\left( y \right) -I_{g}\left( y \right) }{B_{g}\left( y \right) }, \\ \displaystyle \max _{y\in \Omega \left( x \right) }\frac{B_{b}\left( y \right) -I_{b}\left( y \right) }{B_{b}\left( y \right) }. \end{pmatrix} \le 1 \end{aligned} \end{aligned}$$

(A.1)

for each scenic spot $x \in \left[ 0,1 \right]$, they have the following relationship with the ambient light:

$$\begin{aligned} \begin{aligned} B_{\lambda }\left( x \right) -I_{\lambda }\left( x \right) =E_{\lambda }\left( x \right) \cdot t_{\lambda }\left( x \right) ^{d\left( x \right) }\cdot \left( 1-\rho _{\lambda }\left( x \right) \right) \end{aligned} \end{aligned}$$

(A.2)

where $E_{\lambda }\left( x \right)$ is the total incident light of the scenic point x, and its value is always $\ge 0$. $t_{\lambda }\left( x \right) ^{d\left( x \right) }$ is the transmission that we need to derive. According to the Beer-Lambert law [3], its equivalent formula is $t_{\lambda }\left( x \right) ^{d\left( x \right) }=e^{-\beta d\left( x \right) }$, where $d\left( x \right)$ is the distance from the camera to the radiant object and $\beta$ is the light attenuation coefficient, so $t_{\lambda }\left( x \right) ^{d\left( x \right) } \ge 0$. $\rho _{\lambda }\left( x \right)$ is the reflectivity of a scene point, and its value falls in $\left[ 0,1 \right]$. Therefore, we can draw the following conclusion:

$$\begin{aligned} \begin{aligned} B_{\lambda }\left( x \right) \ge I_{\lambda }\left( x \right) \end{aligned} \end{aligned}$$

(A.3)

In order to facilitate understanding, we convert Eq. (A.1) into Eq. (A.4):

$$\begin{aligned} \begin{aligned}&\max \begin{pmatrix} \displaystyle \max _{y\in \Omega \left( x \right) }\frac{I_{r}\left( y \right) -B_{r}\left( y \right) }{1-B_{r}\left( y \right) }, \\ \displaystyle \max _{y\in \Omega \left( x \right) }\frac{B_{g}\left( y \right) -I_{g}\left( y \right) }{B_{g}\left( y \right) }, \\ \displaystyle \max _{y\in \Omega \left( x \right) }\frac{B_{b}\left( y \right) -I_{b}\left( y \right) }{B_{b}\left( y \right) }.\end{pmatrix}\\&\quad = \max \begin{pmatrix} \displaystyle \max _{y\in \Omega \left( x \right) }1-\frac{1-I_{r}\left( y \right) }{1-B_{r}\left( y \right) }, \\ \displaystyle \max _{y\in \Omega \left( x \right) }1-\frac{I_{g}\left( y \right) }{B_{g}\left( y \right) }, \\ \displaystyle \max _{y\in \Omega \left( x \right) }1-\frac{I_{b}\left( y \right) }{B_{b}\left( y \right) }. \end{pmatrix} \end{aligned} \end{aligned}$$

(A.4)

Obviously, for g channel and b channel, their values are always in $\left[ 0,1 \right]$. But for the r channel, we need to discuss it further. For any scenic spot $x^{*}$, we mark its corresponding ambient light point as $x_{0}$. It is clear that:

$$\begin{aligned} \begin{aligned} \max _{y\in \Omega \left( x^{*} \right) }\left( I_{r}\left( y \right) -B_{r}\left( y \right) \right) \le \max _{y\in \Omega \left( x_{0} \right) }\left( I_{r}\left( y \right) -B_{r}\left( y \right) \right) \end{aligned} \end{aligned}$$

(A.5)

Based on the fact that $x_{0}\in \Omega \left( x_{0} \right)$, Eq. (A.5) can be further written as:

$$\begin{aligned} \begin{aligned} \max _{y\in \Omega \left( x^{*} \right) }\left( I_{r}\left( y \right) -B_{r}\left( y \right) \right) \le I_{r}\left( x_{0} \right) -B_{r}\left( x_{0} \right) \le 1-B_{r}\left( x_{0} \right) \end{aligned} \nonumber \\ \end{aligned}$$

(A.6)

Due to the serious attenuation of the light in the red band underwater, there is little residual energy to reach underwater scene, so $0< 1-B_{r}\left( x \right) \le 1$, and we can deduce that:

$$\begin{aligned} \begin{aligned} \frac{\max _{y\in \Omega \left( x^{*} \right) }\left( I_{r}\left( y \right) -B_{r}\left( y \right) \right) }{1-B_{r}\left( x_{0} \right) }\le 1 \end{aligned} \end{aligned}$$

(A.7)

In addition, due to $x^{*}$ and $x_{0}$ are one-to-one correspondences, Eq. (A.8) is equivalent to Eq. (A.7):

$$\begin{aligned} \begin{aligned} \max _{y\in \Omega \left( x^{*} \right) }\frac{I_{r}\left( y \right) -B_{r}\left( y \right) }{1-B_{r}\left( y \right) }\le 1 \end{aligned} \end{aligned}$$

(A.8)

To sum up, when the maximum value of Eq. (A.1) falls in r channel, the value range of Eq. (A.1) is as follows:

$$\begin{aligned} \begin{aligned}&0 \le \max \begin{pmatrix} \displaystyle \max _{y\in \Omega \left( x \right) }\frac{B_{g}\left( y \right) -I_{g}\left( y \right) }{B_{g}\left( y \right) }, \\ \displaystyle \max _{y\in \Omega \left( x \right) }\frac{B_{b}\left( y \right) -I_{b}\left( y \right) }{B_{b}\left( y \right) }. \end{pmatrix} \\&\quad \le \max _{y\in \Omega \left( x^{*} \right) }\frac{I_{r}\left( y \right) -B_{r}\left( y \right) }{1-B_{r}\left( y \right) } \le 1 \end{aligned} \end{aligned}$$

(A.9)

According to the above discussion, the convergence of Eq. (8) is verified.