Explainable recommendations with nonnegative matrix factorization

Abstract

Explicable recommendation system is proved to be conducive to improving the persuasiveness of the recommendation system, enabling users to trust the system more and make more intelligent decisions. Nonnegative Matrix Factorization (NMF) produces interpretable solutions for many applications including collaborative filtering as it’s nonnegativity. However, the latent features make it difficult to interpret recommendation results to users because we don’t know the specific meaning of features that users are interested in and the extent to which the items or users belong to these features. To overcome this difficulty, we develop a novel method called Partially Explainable Nonnegative Matrix Factorization (PE-NMF) by employing explicit data to replace part latent variables of item-feature matrix, by which users can learn more about the features of the items and then to make ideal decisions and recommendations. The objective function of PE-NMF is composed of two parts: one part corresponding to explicit features and the other part is about implicit features. We develop an iterative method to minimize the objective function and derive the iterative update rules, with which the objective function can be proved to be decreasing. Finally, the experiments are executed on Yelp, Amazon and Dianping datasets, and the experimental results demonstrate PE-NMF keeps a high prediction performance on both rating prediction and top-N recommendation that compare to fully explainable nonnegative matrix factorization (FE-NMF), which is obtained by using explicit opinions instead of item-feature matrix. Also PE-NMF holds almost the same recommendation ability as NMF.

Appendix

The objective function $O_{EF}$ in Eq. (3) is established using F-norm formulation, the proof of Theorem 1 is referred to Lee and Seung (1999). Actually, $O_{EF}$ is obtained by fixing ${\textbf {U}}$ in $O_{1}$, in this case, $O_{EF}$ is a linear function that only relates to ${\textbf {U}}$. Therefore, we can minimize $O_{EF}$ by taking the derivative with respect to ${\textbf {U}}$.

Proof of Theorem 1

As

$$\begin{aligned} O_{FE}&=\text{Tr}({\mathcal {P}}_{\Omega }({\textbf {X}}-2r){\mathcal {P}}_{\Omega ^{T}} ({\textbf {X}}^{T})+2r\text{Tr}({\textbf {I}}{\mathcal {P}}_{\Omega ^{T}}({\textbf {V}}_{F} {\textbf {U}}^{T}))+2r^{2}\text{Tr}({\textbf {I}}{} {\textbf {I}}^{T}) \\ {}&-2\text{Tr}({\mathcal {P}}_{\Omega }({\textbf {X}}){\mathcal {P}}_{\Omega ^{T}} ({\textbf {V}}_{F}{} {\textbf {U}}^{T}))+\text{Tr}({\mathcal {P}}_{\Omega }({\textbf {U}} {\textbf {V}}_{F}^{T}){\mathcal {P}}_{\Omega ^{T}}({\textbf {V}}_{F}{} {\textbf {U}}^{T})), \end{aligned}$$

Let $\psi _{ik}$ be the Lagrange multiplier for constraints $u_{ik}\ge 0$, and $\Psi =[\psi _{ik}].$ Then Lagrange function ${\mathcal {L}}_{O_{EF}}$ is

$$\begin{aligned} {\mathcal {L}}_{O_{EF}}&=\text{Tr}({\mathcal {P}}_{\Omega }({\textbf {X}}-2r){\mathcal {P}}_{\Omega ^{T}} ({\textbf {X}}^{T})+2r\text{Tr}({\textbf {I}}{\mathcal {P}}_{\Omega ^{T}}({\textbf {V}}_{F}{} {\textbf {U}}^{T})) +2r^{2}\text{Tr}({\textbf {I}}{} {\textbf {I}}^{T}) \\ {}&-2\text{Tr}({\mathcal {P}}_{\Omega }({\textbf {X}}){\mathcal {P}}_{\Omega ^{T}}({\textbf {V}}_{F} {\textbf {U}}^{T}))+\text{Tr}({\mathcal {P}}_{\Omega }({\textbf {U}}{} {\textbf {V}}_{F}^{T}){\mathcal {P}}_{\Omega ^{T}} ({\textbf {V}}_{F}{} {\textbf {U}}^{T}))+\text{Tr}(\Psi {\textbf {U}}^{T}). \end{aligned}$$

The derivative of ${\mathcal {L}}_{O_{EF}}$ with respect to ${\textbf {U}}$ is:

$$\begin{aligned} \frac{{\mathcal {L}}_{O_{EF}}}{\partial {\textbf {U}}}=\,&-2{\mathcal {P}}_{\Omega } ({\textbf {X}}){\textbf {V}}_{F}+2r{\textbf {I}}{} {\textbf {V}}_{F}+2{\mathcal {P}}_{\Omega } ({\textbf {U}}{} {\textbf {V}}_{F}^{T}){\textbf {V}}_{F}+\Psi . \end{aligned}$$

Using KKT conditions $\psi _{ik}u_{ik}=0$, the following equality for $u_{ik}$ is obtained

$$\begin{aligned} -&({\mathcal {P}}_{\Omega }({\textbf {X}}){\textbf {V}}_{F})_{ik}u_{ik}+r({\textbf {I}} {\textbf {V}}_{F})_{ik}+({\mathcal {P}}_{\Omega }({\textbf {U}}{} {\textbf {V}}_{F}^{T}){\textbf {V}}_{F})_{ik}u_{ik}=0, \end{aligned}$$

which leads to the following updating formula:

$$\begin{aligned} u_{ik}\leftarrow u_{ik}\frac{\Big ({\mathcal {P}}_{\Omega }({\textbf {X}}){\textbf {V}}_{F}\Big )_{ik}}{\Big ({\mathcal {P}}_{\Omega } ({\textbf {U}}{} {\textbf {V}}_{F}^{T}+r){\textbf {V}}_{F}\big )_{ik}}. \end{aligned}$$

Thus, we prove the Theorem 1. $\square$

The objective function $O_{PE}$ in Eq. (5) is established using F-norm formulation, thus $O_{PE}$ is certainly bounded from below by zero. To finish the proof of Theorem 3, we need to indicate $O_{PE}$ is non-increasing under the update steps in Eq. (6). Similar to the procedures described in Lee and Seung (1999), the auxiliary function used in Expectation-Maximization algorithm (Dempster et al. 1977; Saul and Pereira 1997) can also be involved in our proofs. The notion of the auxiliary function is given as follows:

Definition 1

(Dempster et al. 1977) $T(v, v^{'})$ is an auxiliary function for E(v) if the following conditions

$$\begin{aligned} T(v, v^{'})\ge E(v), \quad T(v, v)=E(v) \end{aligned}$$

are satisfied.

It is verified that the auxiliary function is beneficial for proving the following lemma.

Lemma 1

(Dempster et al. 1977; Saul and Pereira 1997) If T is an auxiliary function of E, then E is non-increasing under the update

$$\begin{aligned} v^{t+1}=\arg \min _{v}T(v, v^{t}). \end{aligned}$$

(7)

Proof

Since $v^{t+1}=\arg \min \limits _{v}T(v, v^{t})$, thus for $v^{t}$, we can obtain

$$\begin{aligned} T(v^{t+1}, v^{t})\le T(v^{t}, v^{t}). \end{aligned}$$

While $E(v^{t+1})=T(v^{t+1}, v^{t+1})$, according to Definition 1, we get

$$\begin{aligned} E(v^{t+1})\le T(v^{t+1}, v^{t})\le T(v^{t}, v^{t})=E(v^{t}). \end{aligned}$$

In what follows, we will indicate that the update formulas for ${\textbf {U}}_{1}, {\textbf {U}}_{2}, {\textbf {V}}_{2}$ in Eq. (6) is accurately the update in Eq. (7) by defining the suitable auxiliary functions $T(v, v^{t})$ for Eq. (5).

According to the objective function $O_{PE}$ in Eq. (6), we have

$$\begin{aligned} O_{PE}&=\text{Tr}({\mathcal {P}}_{\Omega }({\textbf {X}}){\mathcal {P}}_{\Omega ^{T}}({\textbf {X}}^{T}) -2r\text{Tr}({\textbf {I}}{} {\textbf {X}}^{T}) +2r\text{Tr}({\textbf {I}}{\mathcal {P}}_{\Omega ^{T}}({\textbf {V}}_{2}{} {\textbf {U}}_{2}^{T}))\\&\quad +2r\text{Tr}({\textbf {I}}{\mathcal {P}}_{\Omega ^{T}}({\textbf {V}}_{1}{} {\textbf {U}}_{1}^{T}))+2r^{2} \text{Tr}({\textbf {I}}{} {\textbf {I}}^{T})-2\text{Tr}({\mathcal {P}}_{\Omega }({\textbf {X}}){\mathcal {P}}_{\Omega ^{T}} ({\textbf {V}}_{1}{} {\textbf {U}}_{1}^{T})) \\&\quad-2\text{Tr}({\mathcal {P}}_{\Omega }({\textbf {X}}){\mathcal {P}}_{\Omega ^{T}}({\textbf {V}}_{2}\ textbf{U}_{2}^{T}))+\text{Tr}({\mathcal {P}}_{\Omega }({\textbf {U}}_{1}{} {\textbf {V}}_{1}^{T}) {\mathcal {P}}_{\Omega ^{T}}({\textbf {V}}_{1}{} {\textbf {U}}_{1}^{T})) \\&\quad+\text{Tr}({\mathcal {P}}_{\Omega }({\textbf {U}}_{2}{} {\textbf {V}}_{2}^{T}){\mathcal {P}}_{\Omega ^{T}} ({\textbf {V}}_{2}{} {\textbf {U}}_{2}^{T})) +\text{Tr}({\mathcal {P}}_{\Omega }({\textbf {U}}_{1}{} {\textbf {V}}_{1}^{T}){\mathcal {P}}_{\Omega ^{T}} ({\textbf {V}}_{2}{} {\textbf {U}}_{2}^{T})) \end{aligned}$$

Taking

$$\begin{aligned} E_{{\textbf {U}}_{1}}&=2\text{Tr}({\mathcal {P}}_{\Omega }(r{\textbf {I}}-{\textbf {X}}){\mathcal {P}}_{\Omega ^{T}} ({\textbf {V}}_{1}{} {\textbf {U}}_{1}^{T}))+\text{Tr}({\mathcal {P}}_{\Omega }({\textbf {U}}_{1}{} {\textbf {V}}_{1}^{T}) {\mathcal {P}}_{\Omega ^{T}}({\textbf {V}}_{1}{} {\textbf {U}}_{1}^{T})) \\ {}&\quad+\text{Tr}({\mathcal {P}}_{\Omega }({\textbf {U}}_{1}{} {\textbf {V}}_{1}^{T}){\mathcal {P}}_{\Omega ^{T}} ({\textbf {V}}_{2}{} {\textbf {U}}_{2}^{T})),\\ E_{{\textbf {U}}_{2}}&=2\text{Tr}({\mathcal {P}}_{\Omega }(r{\textbf {I}}-{\textbf {X}}){\mathcal {P}}_{\Omega ^{T}} ({\textbf {V}}_{2}{} {\textbf {U}}_{2}^{T})) +\text{Tr}({\mathcal {P}}_{\Omega }({\textbf {U}}_{2}{} {\textbf {V}}_{2}^{T}){\mathcal {P}}_{\Omega ^{T}} ({\textbf {V}}_{2}{} {\textbf {U}}_{2}^{T})) \\ {}&\quad+\text{Tr}({\mathcal {P}}_{\Omega }({\textbf {U}}_{1}{} {\textbf {V}}_{1}^{T}){\mathcal {P}}_{\Omega ^{T}} ({\textbf {V}}_{2}{} {\textbf {U}}_{2}^{T})),\\ E_{{\textbf {V}}_{2}}&=2\text{Tr}({\mathcal {P}}_{\Omega }(r{\textbf {I}}-{\textbf {X}}){\mathcal {P}}_{\Omega ^{T}} ({\textbf {V}}_{2}{} {\textbf {U}}_{2}^{T})) +\text{Tr}({\mathcal {P}}_{\Omega }({\textbf {U}}_{2}{} {\textbf {V}}_{2}^{T}){\mathcal {P}}_{\Omega ^{T}} ({\textbf {V}}_{2}{} {\textbf {U}}_{2}^{T})) \\ {}&\quad+\text{Tr}({\mathcal {P}}_{\Omega }({\textbf {U}}_{1}{} {\textbf {V}}_{1}^{T}){\mathcal {P}}_{\Omega ^{T}} ({\textbf {V}}_{2}{} {\textbf {U}}_{2}^{T})), \end{aligned}$$

as the part which are only relevant to ${\textbf {U}}_{1}$, ${\textbf {U}}_{2}$ and ${\textbf {V}}_{2}$ in $O_{P}$, respectively, where $k\in K_{1}$ and $l\in K-K_{1}$. Considering any element $u_{ik}$ in ${\textbf {U}}_{1}$, $u_{il}$ in ${\textbf {U}}_{2}$ and $v_{jl}$ in ${\textbf {V}}_{2}$, we use $E_{{\textbf {U}}_{1}}(u_{ik})$, $E_{{\textbf {U}}_{2}}(u_{il})$ and $E_{{\textbf {V}}_{2}}(v_{jl})$ to denote the parts which includes $u_{ik}$, $u_{il}$ and $v_{jl}$ in Eq. (8), respectively. It is easy to check that the first-order derivatives of $E_{{\textbf {U}}_{ik}}$, $E_{{\textbf {U}}_{ik}}$ and $E_{{\textbf {U}}_{ik}}$ as follows:

$$\begin{aligned} E_{{\textbf {U}}_{1}}^{'}(u_{ik})&=\big (-2{\mathcal {P}}_{\Omega } ({\textbf {X}}){\textbf {V}}_{1}+2r{\textbf {IV}}_{1}+2{\mathcal {P}}_{\Omega } ({\textbf {U}}_{1}{} {\textbf {V}}_{1}^{T}){\textbf {V}}_{1}+2{\mathcal {P}}_{\Omega } ({\textbf {U}}_{2}{} {\textbf {V}}_{2}^{T}){\textbf {V}}_{1}\big )_{ik},\\ E_{{\textbf {U}}_{2}}^{'}(u_{il})&=\big (-2{\mathcal {P}}_{\Omega } ({\textbf {X}}){\textbf {V}}_{2}+2r{\textbf {IV}}_{2}+2{\mathcal {P}}_{\Omega } ({\textbf {U}}_{1}{} {\textbf {V}}_{1}^{T}){\textbf {V}}_{2}+2{\mathcal {P}}_{\Omega } ({\textbf {U}}_{2}{} {\textbf {V}}_{2}^{T}){\textbf {V}}_{2}\big )_{il},\\ E_{{\textbf {V}}_{2}}^{'}(v_{jl})&=\big (-2{\mathcal {P}}_{\Omega ^{T}} ({\textbf {X}}^{T}){\textbf {U}}_{2}+2r{\textbf {I}}^{T}{} {\textbf {U}}_{2}+2{\mathcal {P}}_{\Omega ^{T}} ({\textbf {V}}_{2}{} {\textbf {U}}_{2}^{T}){\textbf {U}}_{2}+2{\mathcal {P}}_{\Omega ^{T}} ({\textbf {V}}_{1}{} {\textbf {U}}_{1}^{T}){\textbf {U}}_{2}\big )_{jl}. \end{aligned}$$

Similarly, the second-order derivatives of $E_{{\textbf {U}}_{1}}(u_{ik})$, $E_{{\textbf {U}}_{2}}(u_{il})$ and $E_{{\textbf {V}}_{2}}(v_{jl})$ are gives as

$$\begin{aligned} E_{{\textbf {U}}_{1}}^{''}(u_{ik})=2\big ({\textbf {V}}_{1}^{T}{} {\textbf {V}}_{1}\big )_{kk}, \quad E_{{\textbf {U}}_{2}}^{''}(u_{il})=2\big ({\textbf {V}}_{2}^{T}{} {\textbf {V}}_{2}\big )_{ll}, \quad E_{{\textbf {V}}_{2}}^{''}(v_{jl})=2\big ({\textbf {U}}_{2}^{T}{} {\textbf {U}}_{2}\big )_{ll}. \end{aligned}$$

$\square$

Since the update rule proposed in this paper is essentially element-wise, it is sufficient to indicate that each $E_{{\textbf {U}}_{1}}(u_{ik}), E_{{\textbf {U}}_{2}}(u_{il}), E_{{\textbf {V}}_{2}}(v_{jl})$ are non-increasing under the update formulas of Eq. (7).

Lemma 2

Function

$$\begin{aligned}T(u_{ik}, u_{ik}^{t})&=E_{{\textbf {U}}_{1}}(u_{ik}^{t})+E_{{\textbf {U}}_{1}}^{'}(u_{ik}^{t})(u_{ik}-u_{ik}^{t}) \nonumber \\ {}&\quad +\frac{\Big ({\mathcal {P}}_{\Omega }({\textbf {U}}_{1} {\textbf {V}}_{1}^{T}+{\textbf {U}}_{2}{} {\textbf {V}}_{2}^{T} +r\big ){\textbf {V}}_{1}\Big )_{ik}}{u_{ik}^{t}}(u_{ik}-u_{ik}^{t})^{2},\nonumber \\&\quad T(u_{il}, u_{il}^{t})=E_{{\textbf {U}}_{2}}(u_{il}^{t})+E_{{\textbf {U}}_{2}}^{'}(u_{il}^{t}) (u_{il}-u_{il}^{t}) \nonumber \\&\quad +\frac{\Big ({\mathcal {P}}_{\Omega }({\textbf {U}}_{1}{} {\textbf {V}}_{1}^{T}+{\textbf {U}}_{2}{} {\textbf {V}}_{2}^{T} +r\big ){\textbf {V}}_{2}\Big )_{il}}{u_{il}^{t}}(u_{il}-u_{il}^{t})^{2},\nonumber \\&\quad T(v_{jl}, v_{jl}^{t})=E_{{\textbf {V}}_{2}}(v_{jl}^{t})+E_{{\textbf {V}}_{2}}^{'}(v_{jl}^{t})(v_{jl}-v_{jl}^{t}) \nonumber \\&\quad +\frac{\Big ({\mathcal {P}}_{\Omega ^{T}}({\textbf {V}}_{1}{} {\textbf {U}}_{1}^{T}+{\textbf {V}}_{2}{} {\textbf {U}}_{2}^{T} +r\big ){\textbf {U}}_{2}\Big )_{jl}}{v_{jl}^{t}}(v_{jl}-v_{jl}^{t})^{2}, \end{aligned}$$

(8)

are auxiliary functions for $E_{{\textbf {U}}_{1}}(u_{ik}), E_{{\textbf {U}}_{2}}(u_{il}), E_{{\textbf {V}}_{2}}(v_{jl})$, respectively, which are only associated with $u_{ik}, u_{il}$ and $v_{jl}$ in $O_{P}$.

Proof

Since $T(u_{ik}^{t}, u_{ik}^{t})=E_{{\textbf {U}}_{1}}(u_{ik}^{t}), T(u_{il}^{t}, u_{il}^{t})=E_{{\textbf {U}}_{2}}(u_{il}^{t})$ and $T(v_{jl}^{t}, v_{jl}^{t})=E_{{\textbf {V}}_{2}}(v_{jl}^{t})$ are obvious, we only need to specify $T(u_{ik}, u_{ik}^{t})\ge E_{{\textbf {U}}_{1}}(u_{ik}^{t}), T(u_{il}, u_{il}^{t})\ge E_{{\textbf {U}}_{2}}(u_{il}^{t})$ and $T(v_{jl}, v_{jl}^{t})\ge E_{{\textbf {V}}_{2}}(v_{jl}^{t})$. To achieve this, we compare the Taylor series expansion of $E_{{\textbf {U}}_{1}}(u_{ik}), E_{{\textbf {U}}_{2}}(u_{il}), E_{{\textbf {V}}_{2}}(v_{jl})$ at $u_{ik}^{t}, u_{il}^{t}$ and $v_{jl}^{t}$, respectively.

$$\begin{aligned} E_{{\textbf {U}}_{1}}(u_{ik})&=E_{{\textbf {U}}_{1}}(u_{ik}^{t})+E_{{\textbf {U}}_{1}}^{'}(u_{ik}^{t})(u_{ik}-u_{ik}^{t}) +\frac{E_{{\textbf {U}}_{1}}^{''}(u_{ik}^{t})}{2}(u_{ik}-u_{ik}^{t})^{2}\\ E_{{\textbf {U}}_{2}}(u_{il})&=E_{{\textbf {U}}_{2}}(u_{il}^{t})+E_{{\textbf {U}}_{2}}^{'}(u_{il}^{t})(u_{il}-u_{il}^{t}) +\frac{E_{{\textbf {U}}_{2}}^{''}(u_{il}^{t})}{2}(u_{il}-u_{il}^{t})^{2}\\ E_{{\textbf {V}}_{2}}(v_{jl})&=E_{{\textbf {V}}_{2}}(v_{jl}^{t})+E_{{\textbf {V}}_{2}}^{'}(v_{jl}^{t})(v_{jl}-v_{jl}^{t}) +\frac{E_{{\textbf {V}}_{2}}^{''}(v_{jl}^{t})}{2}(v_{jl}-v_{jl}^{t})^{2} \end{aligned}$$

with the expression of $T(u_{ik}, u_{ik}^{t})$ to find that $T(u, u_{ik}^{t})\ge E_{{\textbf {U}}_{1}}(u_{ik}^{t})$ is equivalent to

$$\begin{aligned} \frac{\Big ({\mathcal {P}}_{\Omega }({\textbf {U}}_{1}{} {\textbf {V}}_{1}^{T}+{\textbf {U}}_{2}{} {\textbf {V}}_{2}^{T} +r\big ){\textbf {V}}_{1}\Big )_{ik}}{u_{ik}^{t}}\ge \frac{E_{{\textbf {U}}_{1}}^{''}(u_{ik}^{t})}{2} =\big ({\textbf {V}}_{1}^{T}{} {\textbf {V}}_{1}\big )_{kk}, \end{aligned}$$

which follows $\Big ({\mathcal {P}}_{\Omega }({\textbf {U}}_{1}{} {\textbf {V}}_{1}^{T}+{\textbf {U}}_{2}{} {\textbf {V}}_{2}^{T} +r\big ){\textbf {V}}_{1}\Big )_{ik}\ge u_{ik}^{t}\big ({\textbf {V}}_{1}^{T}{} {\textbf {V}}_{1}\big )_{kk}.$ On the other hand, since

$$\begin{aligned} \Big (\big ({\mathcal {P}}_{\Omega }({\textbf {U}}_{1}{} {\textbf {V}}_{1}^{T}\big ){\textbf {V}}_{1}\Big )_{ik} =\sum _{p=1}^{K_{1}}u_{ip}^{t}({\textbf {V}}_{1}^{T}{} {\textbf {V}}_{1})_{pk}\ge u_{ik}^{t}({\textbf {V}}_{1}^{T}{} {\textbf {V}}_{1})_{kk}. \end{aligned}$$

Therefore, $T(u_{ik}, u_{ik}^{t})\ge E_{{\textbf {U}}_{1}}(u_{ik}^{t})$, similar proofs to $T(u_{il}, u_{il}^{t})\ge E_{{\textbf {U}}_{2}}(u_{il}^{t})$ and $T(v_{jl}, v_{jl}^{t})\ge E_{{\textbf {V}}_{2}}(v_{jl}^{t})$. Thus, the results are obtained.

Based on preceding analysis, it is easy to get the proof of Theorem 3. $\square$

Proof of Theorem 3

Replacing $T(u_{ik}, u_{ik}^{t}), T(u_{il}, u_{il}^{t})$ and $T(v_{jl}, v_{jl}^{t})$ in Eq. (8) by Eq. (6) leads to the following update formulas

$$\begin{aligned} u_{ik}^{t+1}=\arg \min _{u_{ik}}T(u_{ik},u_{ik}^{t}), u_{il}^{t+1}=\arg \min _{u_{il}}T(u_{il},u_{il}^{t}), v_{jl}^{t+1}=\arg \min _{v_{jl}}T(v_{jl}, v_{jl}^{t}). \end{aligned}$$

To find $u_{ik}$ subject to $T(u_{ik}, u_{ik}^{'})$ reaches to minimum at $u_{ik}$, set

$$\begin{aligned} \frac{\partial T(u_{ik}, u_{ik}^{'})}{\partial u_{ik}}=\,&E_{{\textbf {U}}_{1}}^{'}(u_{ik}^{t})+2\frac{\Big ({\mathcal {P}}_{\Omega }({\textbf {U}}_{1}\ {\textbf {V}}_{1}^{T}+{\textbf {U}}_{2}{} {\textbf {V}}_{2}^{T}+r\big ){\textbf {V}}_{1}\Big )_{ik}}{u_{ik}^{t}}(u_{ik}-u_{ik}^{t}) =0, \end{aligned}$$

which leads to $u_{ik}=u_{ik}^{t}\frac{\big ({\mathcal {P}}_{\Omega }({\textbf {X}}){\textbf {V}}_{1}\big )_{ik} }{\Big ({\mathcal {P}}_{\Omega }({\textbf {U}}_{1}{} {\textbf {V}}_{1}^{T}+{\textbf {U}}_{2} {\textbf {V}}_{2}^{T}+r\big ){\textbf {V}}_{1}\Big )_{ik}}.$ Thus, $u_{ik}^{t+1}=u_{ik}^{t}\frac{\big ({\mathcal {P}}_{\Omega }({\textbf {X}}){\textbf {V}}_{1}\big )_{il} }{\Big ({\mathcal {P}}_{\Omega }({\textbf {U}}_{1}{} {\textbf {V}}_{1}^{T}+{\textbf {U}}_{2} {\textbf {V}}_{2}^{T}+r\big ){\textbf {V}}_{1}\Big )_{jl}}.$ Similarly, we can prove

$$\begin{aligned} u_{il}^{t+1}&=\arg \min _{u_{il}}T(u_{il}, u_{il}^{'})=u_{il}^{t}\frac{\big ({\mathcal {P}}_{\Omega }({\textbf {X}}){\textbf {V}}_{2}\big )_{il} }{\Big ({\mathcal {P}}_{\Omega }({\textbf {U}}_{1}{} {\textbf {V}}_{1}^{T}+{\textbf {U}}_{2}{} {\textbf {V}}_{2}^ {T}+r\big ){\textbf {V}}_{2}\Big )_{il}},\\ v_{jl}^{t+1}&=\arg \min _{v_{jl}}T(v_{jl}, v_{jl}^{'})=v_{jl}^{t}\frac{\Big ({\mathcal {P}}_{\Omega } ({\textbf {X}}^{T}){\textbf {U}}_{2}\Big )_{jl}}{\Big ({\mathcal {P}}_{\Omega ^{T}}({\textbf {V}}_{1} {\textbf {U}}_{1}^{T}+{\textbf {V}}_{2}{} {\textbf {U}}_{2}^{T}+r\big ){\textbf {U}}_{2}\Big )_{jl} }. \end{aligned}$$

Since Eq. (8) is the auxiliary functions for $E_{{\textbf {U}}_{1}}(u_{ik}), E_{{\textbf {U}}_{2}}(u_{il})$ and $E_{{\textbf {V}}_{2}}(v_{jl})$, which are non-increasing under $u_{ik}^{t+1}, u_{il}^{t+1}$ and $v_{jl}^{t+1}$. Therefore, Eq. (5) is non-increasing under the update rules in Eq. (6). $\square$