Strong convergence of an inertial extrapolation method for a split system of minimization problems

Anteneh Getachew Gebrie; Rabian Wangkeeree

doi:10.1515/dema-2020-0025

Open Access Published by De Gruyter Open Access December 31, 2020

Strong convergence of an inertial extrapolation method for a split system of minimization problems

Anteneh Getachew Gebrie and Rabian Wangkeeree

From the journal Demonstratio Mathematica

https://doi.org/10.1515/dema-2020-0025

Abstract

In this article, we propose an inertial extrapolation-type algorithm for solving split system of minimization problems: finding a common minimizer point of a finite family of proper, lower semicontinuous convex functions and whose image under a linear transformation is also common minimizer point of another finite family of proper, lower semicontinuous convex functions. The strong convergence theorem is given in such a way that the step sizes of our algorithm are selected without the need for any prior information about the operator norm. The results obtained in this article improve and extend many recent ones in the literature. Finally, we give one numerical example to demonstrate the efficiency and implementation of our proposed algorithm.

Keywords: minimization problem; Moreau-Yosida approximate; inertial term; strong convergence

MSC 2010: 65K10; 90C25; 49J52; 47H09

1 Introduction

Throughout this article, unless otherwise stated, we assume that H1, H2 and H are real Hilbert spaces, A:H1→H2 is nonzero bounded linear operator and I denotes the identity operator on a Hilbert space.

Assume Ci (i=1,…,N) and Qi (i=1,…,M) are nonempty closed convex subsets of H1 and H2, respectively. The multiple-set split feasibility problem (MSSFP) which was introduced by Censor et al. [1] is formulated as finding a point

(1)x¯∈⋂i=1NCisuchthatAx¯∈⋂j=1MQj.

In particular, if N=M=1, then the MSSFP (1) is reduced to the problem known as the split feasibility problem (SFP) which was first introduced by Censor and Elfving [2] for modeling inverse problems in finite-dimensional Hilbert spaces. The SFP and MSSFP arise in many fields in the real world, and numerous methods have been proposed to solve the SFP, see for example [3,4,5] and references therein, and MSSFP, see for example [6,7,8] and references therein. Moreover, there are some studies of fixed point problems in the framework of the MSSFP, see for example [9,10,11,12,13,14].

One of the most important problems in optimization theory and nonlinear analysis is the problem of approximating a solution of the unconstrained minimization problem. This can be stated as follows. Find x¯∈H such that

(2)f(x¯)=minx∈Hf(x),

where f:H→ℝ∪{+∞} is proper, lower semicontinuous convex function. Our goal is to introduce a strong convergence iterative algorithm with inertial effect solving the MSSFP (1), where Ci and Qj are solution sets of minimization problems of the form (2) for proper, lower semicontinuous convex functions fi and gj, respectively. We denote by argminf the set of all minimizers of f on H, i.e.,

argminf={x¯∈H:f(x¯)≤f(x),∀x∈H}={x¯∈H:f(x¯)=minx∈Hf(x)}.

If f is a smooth function (mostly if f is twice continuously differentiable), one of the numerical methods for finding approximate solutions of (2) is the Newton method, see [15,16]. Analogous method for solving (2) with better properties for the non-smooth case is based on the notion of proximal mapping introduced by Moreau [17], i.e., the proximal operator of the function f with scaling parameter λ>0 is a mapping proxλf:H→H given by

proxλf(x)=argminy∈Hf(y)+12λ∥x−y∥2.

The minimizers of f (points solving problem (2)) are precisely the fixed points of the proximal operator of f. Thus, solving the optimization problem (2) can be interpreted as finding fixed points of a proximal operator of f and proximal operators are firmly nonexpansive operators. This immediately suggests the most popular method

xn+1=proxλf(xn),

which is called the proximal minimization or the proximal point algorithm introduced by Martinet [18,19] and later by Rockafellar [20].

Let f:H1→ℝ∪{+∞}, g:H2→ℝ∪{+∞} be two proper, convex, lower-semicontinuous functions, where gλ is the Moreau-Yosida approximate [17] of the function g of parameter λ given by gλ(y)=minu∈H2{g(u)+12λ∥y−u∥2}. In [21], Moudafi and Thakur introduced a weakly convergent algorithm solving the following minimization problem:

(3)minx∈H1{f(x)+gλ(Ax)},

in case argminf∩A−1(argming)≠∅. It should be noted that (3) is equivalent to the split minimization problem (SMP): finding a point x¯∈H1 with the property

(4)x¯∈argminfsuchthatAx¯∈argming.

Operator norm is a global invariant and is often difficult to estimate, see for example the Theorem of Hendrickx and Olshevsky in [22]. However, in the several split inverse problem types in the literature, the implementation of the proposed iterative method requires the prior knowledge of operator norm to determine the step sizes. To overcome this difficulty, López et al. [4] introduced a new way of selecting the step sizes for solving the SFP such that the information of the operator norm is not necessary. Moudafi and Thakur [21] used the idea of López et al. [4] to introduce a new way of selecting the step sizes, given by

θλμ(x)=∥A∗(I−proxλg)Ax∥2+∥(I−proxλμf)x∥2

with hλ(x)=12∥(I−proxλg)Ax∥2 and lλμ(x)=12∥(I−proxλμf)x∥2, such that the implementation of the iterative algorithm they proposed for solving (4) does not need any prior information about the operator norm. They proposed the following split proximal algorithm, which generates, from an initial point x1∈H1 assume that xn has been constructed and θλ(xn)≠0, then compute xn+1 via the rule

(5)xn+1=proxλμnf(xn−μnA∗(I−proxλg)Axn),

where step size μn=ρnhλ(xn)+lλμn(xn)θλμn2(xn) with 0<ρn<4 and if θλμn(xn)=0, then xn+1=xn is a solution of SMP (4) and the iterative process stops; otherwise, we set n≔n+1 and go to (5). Based on Moudafi and Thakur [21] many iterative algorithms are proposed for solving SMP (4), see for example those by Abbas et al. in [23], Shehu et al. in [24], Shehu and Iyiola in [25,26,27,28] and Shehu and Ogbuisi in [29].

An inertial term is a two-step iterative method, and the next iterate is defined by making use of the previous two iterates. An inertial extrapolation type algorithm, i.e., an algorithm combining an inertial term, was first introduced by Polyak [30] as an acceleration process in solving a smooth convex minimization problem. It is well known that combining an algorithm with inertial term speeds up or accelerates the rate of convergence of the sequence generated by the algorithm. Consequently, a lot of research interest is now devoted to the inertial extrapolation-type algorithm, see [31,32,33,34] and references therein. Very recently, Shehu and Iyiola [25] proposed an inertial extrapolation-type algorithm for solving the SMP (4) using the setting

l(x)=12∥(I−proxλf)x∥2, ∇l(x)=(I−proxλf)x,
h(x)=12∥(I−proxλg)Ax∥2, ∇h(x)=A∗(I−proxλg)Ax and θ(xn)=∥∇l(x)+∇h(x)∥.

They proposed the following weak convergence result.

Theorem 1.1

Suppose the real parameters{αn}, {βn}and{ρn}satisfy the following conditions:

(c1){αn}is non-increasing sequence and0<δ≤αn≤12,

(c2){βn}is non-increasing sequence and0≤βn≤1−κ3<13for some,κ∈(0,1),

(c3)0<ρn<4, liminfn→∞ρn(4−ρn)>0.

Then the sequence{xn}generated by the iterative algorithm

(6)x0,x1∈H1,zn=xn+βn(xn−xn−1),yn=zn−ρnh(zn)+l(zn)θ2(zn)(∇l(zn)+∇hj(zn)),xn+1=(1−αn)zn+αnyn,

weakly converges to a point x¯solving the SMP (4).

Note that the proximal operator is a natural extension of the notion of a metric projection onto a closed convex set, i.e., proxλf=PQ, where f=δQ (f is the indicator function of a closed convex subset Q of H), and this perspective suggests various properties that we expect proximal operators to obey. However, there is a property that holds for the case of projection operators but not for the case of proximal operators in general. For example, consider a function h defined on H2 given by h(x)=12∥(I−proxλf)Ax∥2, where H1 and H2 are real Hilbert spaces, and f:H2→ℝ∪{+∞} is proper lower semicontinuous convex function. The function h is not differentiable at x=±λ for the case H1=H2=ℝ, A=I and f(x)=|x|, see [26]. However, if f is the indicator function of closed convex subset Q of H2 (f=δQ), then h is convex and weakly lower semicontinuous on H1, and h is always differentiable and ∇h(x)=A∗(I−proxλf)Ax, see [35].

Motivated by the above theoretical views, and inspired by results in [1,21,25], in this article we introduce the strong convergence theorem of an inertial extrapolation-type algorithm that incorporates a proximal operator, a viscosity method and an inertial term to solve the so-called split system of minimization problem (SSMP), given as a task finding a point x¯∈H1 with the property

(7)x¯∈⋂i∈Φ(argminfi)suchthatAx¯∈⋂j∈Ψ(argmingj),

where Φ={1,…,N}, Ψ={1,…,M}, fi:H1→ℝ∪{+∞} and gj:H2→ℝ∪{+∞} are proper, lower semicontinuous convex functions for i∈Φ, j∈Ψ.

Let Γ be the solution set of SSMP (7), i.e.,

Γ=x¯∈⋂i∈Φ(argminfi):Ax¯∈⋂j∈Ψ(argmingj).

Note that if fi=f for all i∈Φ and gj=g for all j∈Ψ, then problem (7) reduces to the SMP (4) that is the problem considered in [21,23,24,25,26,27,28,29]. The aims of this study are twofold: to improve the weak convergence result of an inertial extrapolation-type algorithm proposed by Shehu and Iyiola [25] to a strong convergence result for an approximation of a solution of the SMP (4), and to accelerate and improve the results in [9,10] in solving the SSMP (7).

This article is organized in the following way. In Section 2, we collect some basic and useful definitions, lemmata, and theorems for further study. In Section 3, we propose an iterative method for the SSMP and analyze the strong convergence theorem of the proposed iterative method. In Section 4, we give a numerical example to discuss the performance of the proposed method. Finally, we give some conclusions.

2 Preliminary

In this section, in order to prove our result, we collect some facts and tools in a real Hilbert space H. The symbols “⇀” and “→” denote weak and strong convergence, respectively. Let C be a nonempty closed convex subset of H. The metric projection on C is a mapping PC:H→C defined by

PC(x)=argmin{∥y−x∥:y∈C},x∈H.

Lemma 2.1

Let C be a closed convex subset of H. Givenx∈Hand a pointz∈C, thenz=PC(x)if and only if〈x−z,y−z〉≤0,forally∈C.

Definition 2.1

Let T:H→H. Then,

T is L-Lipschitz if there exists L>0 such that ∥Tx−Ty∥≤L∥x−y∥,∀x,y∈H. If L∈(0,1), then we call T a contraction with constant L. If L=1, then T is called a nonexpansive mapping.
T is firmly nonexpansive if
∥Tx−Ty∥2≤∥x−y∥2−∥(I−T)x−(I−T)y∥2,forallx,y∈H,
which is equivalent to ∥Tx−Ty∥2≤〈Tx−Ty,x−y〉,forallx,y∈H.
If T is firmly nonexpansive, I−T is also firmly nonexpansive.
T is strongly monotone if there exists a constant α>0 such that
〈Tx−Ty,x−y〉≥α∥x−y∥2,forallx,y∈H.
T is inverse strongly monotone if there exists a constant α>0 such that

〈Tx−Ty,x−y〉≥α∥Tx−Ty∥2,forallx,y∈H.

Lemma 2.2

For a real Hilbert space H, we have

∥x+y∥2=∥x∥2+∥y∥2+2〈x,y〉,forallx,y∈H;
∥x+y∥2=∥x∥2+2〈y,x+y〉,forallx,y∈H;
〈x,y〉≤12∥x∥2+12∥y∥2−12∥x−y∥2,forallx,y∈H.

A set-valued mapping T:H→2H is called monotone if, for all x,y∈H, z∈Tx and w∈Ty imply 〈x−y,z−w〉≥0. A monotone mapping T:H→2H is maximal if its graph G(F)={(x,y):y∈F(x),x∈H} is not properly contained in the graph of any other monotone mapping. It is known that a monotone mapping T is maximal if, and only if, for all (x,z)∈H×H, 〈x−y,z−w〉≥0 for all (y,w)∈G(T), implies z∈Tx. If T:H→2H is a maximal monotone set-valued mapping, then we define the resolvent operator JλT associated with T and λ>0 as follows:

(8)JλT(x)=(I+λT)−1(x),x∈H.

It is well known that JλT is single-valued, nonexpansive (see, for example [37,36]) and 1-inverse strongly monotone (firmly nonexpansive). Moreover, 0∈T(x¯) if and only if x¯ is a fixed point of the resolvent operator JλT for all λ>0; see [38].

Let f:H→ℝ∪{+∞} be a proper lower semicontinuous convex function. The domain of f is denoted by dom f; that is, dom f={x∈H:f(x)<∞}. We denote the subdifferential of f at x∈H by ∂f(x), and is given by ∂f(x)={y∈H:f(z)≥f(x)+〈y,z−x〉,∀z∈H}. If ∂f(x)≠∅, f is said to be subdifferentiable at x. It is notable that a point x¯∈H minimizes f if and only if 0∈∂f(x¯). It is the classical result in operator theory that the subdifferential ∂f is a maximal monotone operator and proxλf=(I+λ∂f)−1, namely, for x∈H we have the following equivalence between the subdifferential and proximal operator:

proxλf(x)=y⇔x−y∈λ∂f(y).

Consequently, a point x¯ minimizes f if and only if proxλf(x¯)=x¯. Hence, the convex minimization problem (2) can be formulated as finding fixed point of proximal operator.

Lemma 2.3

[39] Let{cn}and{γn}be sequences of nonnegative real numbers,{βn}be a sequence of real numbers such that

cn+1≤(1−αn)cn+βn+γn,n≥1,

where0<αn<1and∑γn<∞.

Ifβn≤αnMfor someM≥0, then{cn}is a bounded sequence.
If∑αn=∞andlimsupn→∞βnαn≤0, thencn→0asn→∞.

Definition 2.2

Let {Γn} be a real sequence. Then we say {Γn} decrease at infinity if there exists n0∈ℕ such that Γn+1≤Γn for n≥n0. In other words, the sequence {Γn} does not decrease at infinity if there exists a subsequence {Γnt}t≥1 of {Γn} such that Γnt<Γnt+1 for all t≥1.

Lemma 2.4

[40] Let{Γn}be a sequence of real numbers that does not decrease at infinity. Also consider the sequence of integers{φ(n)}n≥n0defined byφ(n)=max{k∈ℕ:k≤n,Γk≤Γk+1}.Then{φ(n)}n≥n0is a nondecreasing sequence verifyinglimn→∞φ(n)=∞, and for alln≥n0, the following two estimates hold:

Γφ(n)≤Γφ(n)+1andΓn≤Γφ(n)+1.

Let D be a nonempty closed convex subset of H. Then we say that the bifunction h:D×D→ℝ satisfies Condition CO on D if the following assumptions are satisfied:

(A1) h(u,u)=0, for all u∈D;

(A2) h is monotone, i.e., h(u,v)+h(v,u)≤0, for all u,v∈D;

(A3) for each u,v,w∈D, limsupα↓0h(αw+(1−α)u,v)≤h(u,v);

(A4) h(u,.) is convex and lower semicontinuous on D for each u∈D.

Lemma 2.5

[41] Let D be a nonempty closed convex subset of H and the bifunctionh:D×D→ℝsatisfies Condition CO on D. Then, for eachr>0andu∈H, there existsw∈Dsuch that

h(w,v)+1r〈v−w,w−u〉≥0,forallv∈D.

The following lemma was given by Combettes and Hirstoaga in [42].

Lemma 2.6

[42] If D is a nonempty closed convex subset of H andh:D×D→ℝis a bifunction satisfying Condition CO on D, then for eachr>0andu∈H, the mappingTrh:H→ℝ(called the resolvent of h), given by

Trh(u)={w∈D:h(w,v)+1r〈v−w,w−u〉≥0,∀v∈D}

satisfies the following conditions:

Trhis single-valued and firmly nonexpansive;
Fix(Trh)={x¯∈D:h(x¯,y)≥0,∀y∈D}, where Fix(Trh)is the set of fixed points ofTrh;
{x¯∈D:h(x¯,y)≥0,∀y∈D}is closed and convex.

3 Main result

First we extend the settings introduced by Moudafi and Thakur [21]. Let λ>0. Then, for x∈H1,

for each i∈Φ, define li(x)=12∥(I−proxλfi)x∥2and∇li(x)=(I−proxλfi)x,
l(x)=lix(x) and ∇l(x)=∇lix(x), where ix∈argmax{li(x):i∈Φ}, i.e., l(x)=max{li(x):i∈Φ},
for each j∈Ψ, define hj(x)=12∥(I−proxλgj)Ax∥2and∇hj(x)=A∗(I−proxλgj)Ax,
for each j∈Ψ, define θj(x)=max{∥∇hj(x)∥,∥∇l(x)∥}.

Remark

From (I)–(IV) given above, ∥∇li(x)∥≤∥∇lix(x)∥=∥∇l(x)∥,li(x)=12∥∇li(x)∥2,∥∇l(x)∥≤θj(x)and∥∇hj(x)∥≤θj(x)foralli∈Φ and forallj∈Ψ.

Consider the parameter sequences satisfying the following conditions.

Assumption 1

Suppose {αn}, {εn}, {ρn}, {ξn(j)}(j∈Ψ) be real sequences satisfying the following conditions:

(C1) 0<αn<1, limn→∞αn=0 and ∑n=1∞αn=∞;

(C2) εn>0 and εn=o(αn);

(C3) 0<ξ≤ξn(j)≤1 and ∑j∈Ψξn(j)=1 for each n≥1;

(C4) 0<ρn<2 and liminfn→∞ρn(2−ρn)>0.

We have plenty of choices for αn and εn satisfying Conditions (C1) and (C2) of Assumption 1. For example, take αn=12n, εn=1n2. Thus, 0<αn<1, limn→∞αn=0 and limn→∞εnαn=0 (i.e., εn=o(αn)).

Using ∇li, li, l, ∇l, hj, ∇hj, θj given in (I)–(IV) and step sizes given in Assumption 1, we are now in a position to state our inertial extrapolation-type algorithm and prove its strong convergence to the solution of the SSMP (7) assuming that solution set Γ is nonempty.

Algorithm 1

Initialization: Let V:H1→H1 be a contraction mapping with constant γ. Choose x0, x1∈H1. Take arbitrary real numbers β and Θ^ such that 0≤β<1 and Θ^>0. Let {αn}, {εn}, {ρn}, {ξn(j)} (j∈Ψ) be real sequences satisfying Assumption 1.

Step 1. Given the iterates xn−1 and xn (n≥1), choose βn such that 0≤βn≤β¯n, where

β¯n≔minβ,εn∥xn−1−xn∥,ifxn−1≠xn,β,otherwise.

Step 2. Evaluate yn=xn+βn(xn−xn−1).

Step 3. For each j∈Ψ find l(yn), hj(yn), θj(yn) and Ψn={j∈Ψ:θj(yn)≠0}.

Step 4. For each j∈Ψ evaluate μn(j)=ρnhj(yn)+l(yn)Θj2(yn), where

Θj(yn)=Θ^,ifj∉Ψn,θj(yn),ifj∈Ψn.

Step 5. Evaluate

zn=yn−12∑j∈Ψ{ξn(j)μn(j)(∇hj(yn)+∇l(yn))}.

Step 6. Evaluate xn+1=αnV(yn)+(1−αn)zn.

Step 7. Set n≔n+1 and go to Step 1.

Remark

From Assumption 1 and Step 1 of Algorithm 1, we have that βnαn∥xn−xn−1∥→0,n→∞. Since {αn} is bounded, we also have βn∥xn−xn−1∥→0,n→∞.

Note that Step 1 of Algorithm 1 is easily implemented in numerical computation since the value of ∥xn−xn−1∥ is a prior known before choosing βn.

Remark

If Ψn=∅, then

max{∥∇hj(yn)∥,∥∇l(yn)∥}=0⇔∥∇hj(yn)∥=0=∥∇l(yn)∥,forallj∈Ψ,⇔∥∇hj(yn)∥=0=∥∇li(yn)∥,foralli∈Φ,j∈Ψ,⇔A∗(I−proxλgj)Ayn=0=(I−proxλfi)yn,foralli∈Φ,j∈Ψ.

Remark

The solution set Γ of problem (7) is closed convex set, because the set of minimizers of any proper, lower semicontinuous function is closed convex and A is bounded linear operator. Therefore, the metric projection PΓ is well defined as we also assume that Γ is nonempty.

Lemma 3.1

For the sequences{xn}, {yn}and{zn}generated by Algorithm 1, we have

∥zn−xˆ∥2≤∥yn−xˆ∥2+ρn(ρn−2)∑j∈Ψξn(j)(hj(yn)+l(yn))2Θj2(yn)

for allxˆ∈Γ. Moreover,{xn}, {yn}and{zn}are bounded sequences.

Proof

Let xˆ∈Γ. From the definition of yn, we get

(9)∥yn−xˆ∥=∥xn+βn(xn−xn−1)−xˆ∥≤∥xn−xˆ∥+βn∥xn−xn−1∥.

Since proxλfi and proxλgj are firmly nonexpansive, I−proxλfi and I−proxλgj are also firmly nonexpansive, and since xˆ verifies (7) (since minimizers of any function are exactly fixed points of its proximal mapping), we have for all x∈H1

(10)〈∇l(x),x−xˆ〉=〈∇lix(x),x−xˆ〉=〈(I−proxλfix)x,x−xˆ〉≥∥I−proxλfixx∥2=2lix(x)=2l(x)

and

(11)〈∇hj(x),x−xˆ〉=〈A∗(I−proxλgj)Ax,x−xˆ〉=〈(I−proxλgj)Ax,Ax−Axˆ〉≥∥(I−proxλgj)Ax∥2=2hj(x),forallj∈Ψ.

Using the definition of zn and Lemma 2.2 (i), we have

(12)∥zn−xˆ∥2=yn−12∑j∈Ψ{ξn(j)μn(j)(∇hj(yn)+∇l(yn))}−xˆ2≤∥yn−xˆ∥2+12∑j∈Ψ{ξn(j)μn(j)(∇hj(yn)+∇l(yn))}2−∑j∈Ψ{ξn(j)μn(j)(∇hj(yn)+∇l(yn))},yn−xˆ.

Noting ∥∇hj(yn)∥≤Θj(yn) and ∥∇l(yn)∥≤Θj(yn) and using the convexity of ∥.∥2, we have

(13)12∑j∈Ψ{ξn(j)μn(j)(∇hj(yn)+∇l(yn))}2≤12∑j∈Ψξn(j)μn(j)∇l(yn)2+12∑j∈Ψξn(j)μn(j)∇hj(yn)2≤12∑j∈Ψξn(j)∥μn(j)∇l(yn)∥2+12∑j∈Ψξn(j)∥μn(j)∇hj(yn)∥2=12∑j∈Ψξn(j)(μn(j))2∥∇l(yn)∥2+12∑j∈Ψξn(j)(μn(j))2∥∇hj(yn)∥2=12∑j∈Ψξn(j)ρnhj(yn)+l(yn)Θj2(yn)2(∥∇l(yn)∥2+∥∇hj(yn)∥2)≤∑j∈Ψξn(j)ρnhj(yn)+l(yn)Θj2(yn)2Θj2(yn)=ρn2∑j∈Ψξn(j)(hj(yn)+l(yn))2Θj2(yn).

From (10) and (11), we have

(14)∑j∈Ψ{ξn(j)μn(j)(∇hj(yn)+∇l(yn))},yn−xˆ=∑j∈Ψ{ξn(j)μn(j)(〈∇l(yn),yn−xˆ〉+〈∇hj(yn),yn−xˆ〉)}≥∑j∈Ψ{ξn(j)μn(j)(2l(yn)+2hj(yn))}=∑j∈Ψ{ξn(j)ρnhj(yn)+l(yn)Θj2(yn)(2l(yn)+2hj(yn))}=2ρn∑j∈Ψξn(j)(hj(yn)+l(yn))2Θj2(yn).

In view of (12), (13) and (14), we have

(15)∥zn−xˆ∥2≤∥yn−xˆ∥2+ρn2∑j∈Ψξn(j)(hj(yn)+l(sn))2Θj2(yn)−2ρn∑j∈Ψξn(j)(hj(yn)+l(yn))2Θj2(yn)=∥yn−xˆ∥2+ρn(ρn−2)∑j∈Ψξn(j)(hj(yn)+l(yn))2Θj2(yn).

Next show that the sequences {xn}, {yn} and {zn} are bounded. From (15) and (C4) of Assumption 1, we have

(16)∥zn−xˆ∥≤∥yn−xˆ∥.

Using (9), (16) and the definition of xn+1, we get

(17)∥xn+1−xˆ∥=∥αnV(yn)+(1−αn)zn−xˆ∥=∥(1−αn)(zn−xˆ)+αn(V(yn)−V(xˆ))+αn(V(xˆ)−xˆ)∥≤(1−αn)∥zn−xˆ∥+αn∥V(yn)−V(xˆ)∥+αn∥V(xˆ)−xˆ∥=(1−αn)∥zn−xˆ∥+αnγ∥yn−xˆ∥+αn∥V(xˆ)−xˆ∥≤(1−αn(1−γ))∥yn−xˆ∥+αn∥V(xˆ)−xˆ∥≤(1−αn(1−γ))∥xn−xˆ∥+(1−αn(1−γ))βn∥xn−xn−1∥+αn∥V(xˆ)−xˆ∥≤(1−αn(1−γ))∥xn−xˆ∥+αn(1−γ)(1−αn(1−γ))1−γβnαn∥xn−xn−1∥+∥V(xˆ)−xˆ∥1−γ.

Observe that by (C1) of Assumption 1 and Remark 3, we see that

limn→∞(1−αn(1−γ))1−γβnαn∥xn−xn−1∥=0.

Let

M=2max∥V(xˆ)−xˆ∥1−γ,supn≥1(1−αn(1−γ))1−γβnαn∥xn−xn−1∥.

Then (17) becomes

∥xn+1−xˆ∥≤(1−αn(1−γ))∥xn−xˆ∥+αn(1−γ)M.

Thus, by Lemma 2.3 the sequence {xn} is bounded. As a consequence, {yn}, {V(yn)} and {zn} are also bounded.□

We now have the following strong convergence theorem for an approximation of solution of a Problem (7).

Theorem 3.2

The sequence{xn}generated by Algorithm 1 converges strongly tox¯∈Γ, wherex¯=PΓV(x¯).

Proof of Theorem 3.2

Claim 1: There exists a uniquex¯∈H1such thatx¯=PΓV(x¯).

As a result of

∥PΓV(x)−PΓV(y)∥≤∥V(x)−V(y)∥≤γ∥x−y∥,forallx,y∈H1,

the mapping PΓV is a contraction mapping of H1 into itself. Hence, by the Banach contraction principle there exists a unique element x¯∈H1 such that x¯=PΓV(x¯). Clearly, x¯∈Γ and we have

x¯=PΓV(x¯)⇔〈x¯−V(x¯),y−x¯〉≥0,forally∈Γ.

Claim 2:The sequence{xn}converges strongly tox¯∈Γ, wherex¯=PΓV(x¯).

Let x¯∈Γ, where x¯=PΓV(x¯). Now

(18)∥yn−x¯∥2=∥xn+βn(xn−xn−1)−x¯∥2=∥xn−x¯∥2+βn2∥xn−xn−1∥2+2βn〈xn−x¯,xn−xn−1〉.

From Lemma 2.2(iii), we have

(19)〈xn−x¯,xn−xn−1〉=12∥xn−x¯∥2−12∥xn−1−x¯∥2+12∥xn−xn−1∥2.

From (18) and (19) and since 0≤βn<1, we get

(20)∥yn−x¯∥2=∥xn−x¯∥2+βn2∥xn−xn−1∥2+βn(∥xn−x¯∥2−∥xn−1−x¯∥2+∥xn−xn−1∥2)≤∥xn−x¯∥2+2βn∥xn−xn−1∥2+βn(∥xn−x¯∥2−∥xn−1−x¯∥2).

Using the definition of xn+1 and Lemma 2.2(ii), we have

(21)∥xn+1−x¯∥2=∥αnV(yn)+(1−αn)zn−x¯∥2=∥αn(V(yn)−x¯)+(1−αn)(zn−x¯)∥2=(1−αn)∥zn−x¯∥2+2αn〈V(yn)−x¯,xn+1−x¯〉.

Lemma 3.1 together with (20) and (21) gives

(22)∥xn+1−x¯∥2=(1−αn)∥zn−x¯∥2+2αn〈V(yn)−x¯,xn+1−x¯〉≤(1−αn)∥yn−x¯∥2+2αn〈V(yn)−x¯,xn+1−x¯〉+(1−αn)ρn(ρn−2)∑j∈Ψξn(j)(hj(yn)+l(yn))2Θj2(yn)≤(1−αn)∥xn−x¯∥2+2(1−αn)βn∥xn−xn−1∥2+(1−αn)βn(∥xn−x¯∥2−∥xn−1−x¯∥2)+2αn〈V(yn)−x¯,xn+1−x¯〉+(1−αn)ρn(ρn−2)∑j∈Ψξn(j)(hj(yn)+l(yn))2Θj2(yn).

Since the sequences {xn} and {V(yn)} are bounded, there exists M1 such that 2〈V(yn)−x¯,xn+1−x¯〉≤M1 for all n≥1. Thus, from (22), we obtain

(23)∥xn+1−x¯∥2≤(1−αn)∥xn−x¯∥2+2(1−αn)βn∥xn−xn−1∥2+(1−αn)βn(∥xn−x¯∥2−∥xn−1−x¯∥2)+αnM1+(1−αn)ρn(ρn−2)∑j∈Ψξn(j)(hj(yn)+l(yn))2Θj2(yn).

Let us distinguish the following two cases related to the behavior of the sequence {Γn}, where Γn=∥xn−x¯∥2.

Case 1. Suppose the sequence {Γn} decreases at infinity. Thus, there exists n0∈ℕ such that Γn+1≤Γn for n≥n0. Then, {Γn} converges and Γn−Γn+1→0 as n→0.

From (23) we have

(24)(1−αn)ρn(2−ρn)∑j∈Ψξn(j)(hj(yn)+l(yn))2Θj2(yn)≤(Γn−Γn+1)+αnM1+(1−αn)βn(Γn−Γn−1)+2(1−αn)βn∥xn−xn−1∥2.

Since Γn−Γn+1→0 alternatively Γn−1−Γn→0 and using (C1) of Assumption 1 and Remark 3 (noting αn→0, 0<αn<1, βn∥xn−xn−1∥→0 and {xn} is bounded), we have from (24)

(25)(1−αn)ρn(2−ρn)∑j∈Ψξn(j)(hj(yn)+l(yn))2Θj2(yn)→0,n→∞.

Conditions (C1) and (C4) of Assumption 1 (i.e., 0<αn<1, αn→0 and liminfn→∞ρn(2−ρn)>0) together with (25) yield

(26)∑j∈Ψξn(j)(hj(yn)+l(yn))2Θj2(yn)→0,n→∞.

In view of (26) and the restriction condition imposed on ξn(j) in (C3) of Assumption 1, we have

(27)(hj(yn)+l(yn))2Θj2(yn)→0,n→∞

for all j∈Ψ.

Now, using the definition of zn and the convexity of ∥.∥2, we have

(28)∥yn−zn∥2=12∑j∈Ψξn(j)μn(j)(∇hj(yn)+∇l(yn))2≤12∑j∈Ψξn(j)μn(j)∇hj(yn)2+12∑j∈Ψξn(j)μn(j)∇l(yn)2≤12∑j∈Ψξn(j)(μn(j))2∥∇hj(yn)∥2+12∑j∈Ψξn(j)(μn(j))2∥∇l(yn)∥2=12∑j∈Ψ{ξn(j)(μn(j))2(∥∇hj(yn)∥2+∥∇l(yn)∥2)}≤∑j∈Ψξn(j)(μn(j))2Θj2(yn)=∑j∈Ψξn(j)ρnhj(yn)+l(yn)Θj2(yn)2Θj2(yn)=ρn2∑j∈Ψξn(j)(hj(yn)+l(yn))2Θj2(yn)≤4∑j∈Ψξn(j)(hj(yn)+l(yn))2Θj2(yn).

Thus, (28) together with (26) gives

(29)∥yn−zn∥→0,n→∞.

Moreover, using the definition of yn and Remark 3, we have

(30)∥xn−yn∥=∥xn−xn−βn(xn−xn−1)∥=βn∥xn−xn−1∥→0,n→∞.

By (29) and (30), we get

(31)∥xn−zn∥≤∥xn−yn∥+∥yn−zn∥→0,n→∞.

Using the definition of xn+1, (C1) of Assumption 1 and noting that {V(yn)} and {zn} are bounded, we have The results from (31) and (32) give

(33)∥xn+1−xn∥≤∥xn+1−zn∥+∥zn−xn∥→0,n→∞.

For each i∈Φ and for each j∈Ψ, ∇hj(.) and ∇li(.) are Lipschitz continuous with constant ∥A∥2 and 1, respectively. Since the sequence {yn} is bounded and

∥∇hj(yn)∥=∥∇hj(yn)−∇hj(x¯)∥≤∥A∥2∥yn−x¯∥,forallj∈Ψ,∥∇li(yn)∥=∥∇li(yn)−∇li(x¯)∥≤∥yn−x¯∥,foralli∈Φ,

we have the sequences {∥∇li(yn)∥}n=1∞ and {∥∇hj(yn)∥}n=1∞ are bounded for each i∈Φ and j∈Ψ. Hence, the boundedness of {∥∇li(yn)∥}n=1∞ for all i∈Φ gives {∥∇l(yn)∥}n=1∞ is a bounded. Thus, we have {θj2(yn)}n=1∞ is bounded for each j∈Ψ. Therefore, the sequence {Θj2(yn)}n=1∞ is bounded sequence for each j∈Ψ. Consequently, using (27), we have

limn→∞(hj(yn)+l(yn))=0⇔limn→∞hj(yn)=limn→∞l(yn)=0,forallj∈Ψ.

By definition of l(yn), we have li(yn)≤l(yn),foralli∈Φ. Therefore,

limn→∞hj(yn)=limn→∞li(yn)=0,∀i∈Φ,forallj∈Ψ.

Let p be a weak cluster point of {xn}, there exists a subsequence {xnk} of {xn} such that xnk⇀p as k→∞. Then, in view of xnk⇀p and (30), we also see that ynk⇀p. The weak lower-semicontinuity of hj(.) implies that

0≤hj(p)≤liminfk→∞hj(ynk)=limn→∞hj(yn)=0,forallj∈Ψ.

That is, hj(p)=12∥(I−proxλgj)Ap∥2=0 for all j∈Ψ, i.e., Ap is a fixed point of the proximal mapping of each gj, or equivalently, 0∈∂gj(Ap) for all j∈Ψ. In other words, Ap is a minimizer of each gj for all j∈Ψ.

Likewise, the weak lower-semicontinuity of li(.) implies that

0≤li(p)≤liminfk→∞li(ynk)=limn→∞li(yn)=0,foralli∈Φ.

That is, li(p)=12∥(I−proxλfi)p∥2=0 for all i∈Φ, i.e., p is a fixed point of the proximal mapping of each fi or equivalently, 0∈∂fi(p) for all i∈Φ. In other words, p is a minimizer of each fi for all i∈Φ. Thus, p∈Γ.

Next, we show that limsupn→∞〈(I−V)x¯,x¯−xn〉≤0. Indeed, since x¯=PΓV(x¯) and p∈Γ we obtain

(34)limsupn→∞〈(I−V)x¯,x¯−xn〉=limk→∞〈(I−V)x¯,x¯−xnk〉=〈(I−V)x¯,x¯−p〉≤0.

Since ∥xn+1−xn∥→0 from (33), by (34) we have

limsupn→∞〈(I−V)x¯,x¯−xn+1〉≤0.

Using Lemma 3.1 (the fact that ∥zn−x¯∥≤∥yn−x¯∥, i.e., from (16)), we have

(35)∥xn+1−x¯∥2=〈αnV(yn)+(1−αn)zn−x¯,xn+1−x¯〉=αn〈V(yn)−V(x¯),xn+1−x¯〉+(1−αn)〈zn−x¯,xn+1−x¯〉+αn〈V(x¯)−x¯,xn+1−x¯〉≤γαn∥yn−x¯∥∥xn+1−x¯∥+(1−αn)∥zn−x¯∥∥xn+1−x¯∥+αn〈V(x¯)−x¯,xn+1−x¯〉≤(1−αn(1−γ))∥yn−x¯∥∥xn+1−x¯∥+αn〈V(x¯)−x¯,xn+1−x¯〉≤(1−αn(1−γ))∥yn−x¯∥22+∥xn+1−x¯∥22+αn〈V(x¯)−x¯,xn+1−x¯〉.

Therefore, from (35), we have

(36)∥xn+1−x¯∥2≤1−αn(1−γ)1+αn(1−γ)∥yn−x¯∥2+2αn1+αn(1−γ)〈V(x¯)−x¯,xn+1−x¯〉=1−2αn(1−γ)1+αn(1−γ)∥yn−x¯∥2+2αn1+αn(1−γ)〈V(x¯)−x¯,xn+1−x¯〉.

Combining (36) and

∥yn−x¯∥=∥xn+βn(xn−xn−1)−x¯∥≤∥xn−x¯∥+βn∥xn−xn−1∥,

it follows that

(37)∥xn+1−x¯∥2≤1−2αn(1−γ)1+αn(1−γ)(∥xn−x¯∥+βn∥xn−xn−1∥)2+2αn1+αn(1−γ)〈V(x¯)−x¯,xn+1−x¯〉=1−2αn(1−γ)1+αn(1−γ)(∥xn−x¯∥2+βn2∥xn−xn−1∥2+2βn∥xn−x¯∥∥xn−xn−1∥)+2αn1+αn(1−γ)〈V(x¯)−x¯,xn+1−x¯〉≤1−2αn(1−γ)1+αn(1−γ)∥xn−x¯∥2+βn2∥xn−xn−1∥2+2βn∥xn−x¯∥∥xn−xn−1∥+2αn1+αn(1−γ)〈V(x¯)−x¯,xn+1−x¯〉.

Since {xn} is bounded there exists M2>0 such that ∥xn−x¯∥≤M2 for all n≥1. Thus, in view of (37), we have

(38)Γn+1≤1−2αn(1−γ)1+αn(1−γ)Γn+βn∥xn−xn−1∥(βn∥xn−xn−1∥+2M2)+2αn1+αn(1−γ)〈V(x¯)−x¯,xn+1−x¯〉=(1−δn)Γn+δnϑn,

where δn=2αn(1−γ)1+αn(1−γ) and

ϑn=1+αn(1−γ)2(1−γ)βnαn∥xn−xn−1∥{βn∥xn−xn−1∥+2M2}+11−γ〈V(x¯)−x¯,xn+1−x¯〉.

From (35), (C1) of Assumption 1 and Remark 3, we have ∑n=1∞δn=∞ and limsupn→∞ϑn≤0. Thus, using Lemma 2.3 and (38), we get Γn→0 as n→∞. Hence, xn→x¯ as n→∞.

Case 2. Assume that {Γn} does not decrease at infinity. Let φ:ℕ→ℕ be a mapping for all n≥n0 (for some n0 large enough) defined by

φ(n)=max{k∈ℕ:k≤n,Γk≤Γk+1}.

By Lemma 2.4, {φ(n)}n=n0∞ is a nondecreasing sequence, φ(n)→∞ as n→∞ and

(39)Γφ(n)≤Γφ(n)+1andΓn≤Γφ(n)+1,foralln≥n0.

In view of ∥xφ(n)−x¯∥2−∥xφ(n)+1−x¯∥2=Γφ(n)−Γφ(n)+1≤0 for all n≥n0 and (23), we have for all n≥n0

(40)(1−αφ(n))ρφ(n)(2−ρφ(n))∑j∈Ψξφ(n)(j)(hj(yφ(n))+l(yφ(n)))2Θj2(yφ(n))≤(Γφ(n)−Γφ(n)+1)+αφ(n)M1+(1−αφ(n))βφ(n)(Γφ(n)−Γφ(n)−1)+2(1−αφ(n))βφ(n)∥xφ(n)−xφ(n)−1∥2≤αφ(n)M1+(1−αφ(n))βφ(n)(Γφ(n)−Γφ(n)−1)+2(1−αφ(n))βφ(n)∥xφ(n)−xφ(n)−1∥2≤αφ(n)M1+(1−αφ(n))βφ(n)∥xφ(n)−xφ(n)−1∥(Γφ(n)+Γφ(n)−1+2(1−αφ(n))βφ(n)∥xφ(n)−xφ(n)−1∥2.

Thus, from (40) together with (C1) and (C2) from Assumption 1 and Remark 3, we have for each j∈Ψ,

(41)(hj(yφ(n))+l(yφ(n)))2Θj2(yφ(n))→0,n→∞.

Using a similar procedure to that in Case 1, we have

limn→∞∥xφ(n)−yφ(n)∥=limn→∞∥xφ(n)+1−xφ(n)∥=0.

Since {xφ(n)} is bounded, there exists a subsequence of {xφ(n)}, still denoted by {xφ(n)} which converges weakly to p. By a similar argument to that in Case 1, we conclude immediately that p∈Γ. In addition, by the similar argument to that in Case 1, we have limsupn→∞〈(I−V)x¯,x¯−xφ(n)〉≤0. Since limn→∞∥xφ(n)+1−xφ(n)∥=0, we get limsupn→∞〈(I−V)x¯,x¯−xφ(n)+1〉≤0. From (38), we have

(42)Γφ(n)+1≤(1−δφ(n))Γφ(n)+δφ(n)ϑφ(n),

where δφ(n)=2αφ(n)(1−γ)1+αφ(n)(1−γ) and

ϑφ(n)=1+αφ(n)(1−γ)2(1−γ)βφ(n)αφ(n)∥xφ(n)−xφ(n)−1∥{βφ(n)∥xφ(n)−xφ(n)−1∥+2M2}+11−γ〈V(x¯)−x¯,xφ(n)+1−x¯〉.

Using Γφ(n)−Γφ(n)+1≤0 for all n≥n0 and ϑφ(n)>0, the last inequality gives

0≤−δφ(n)Γφ(n)+δφ(n)ϑφ(n).

Since δφ(n)>0, we obtain ∥xφ(n)−x¯∥2=Γφ(n)≤ϑφ(n). Moreover, since limsupn→∞ϑφ(n)≤0, we have limn→∞∥xφ(n)−x¯∥=0. Thus, limn→∞∥xφ(n)−x¯∥=0 together with limn→∞∥xφ(n)+1−xφ(n)∥=0 gives limn→∞Γφ(n)+1=0. Therefore, from (39), we obtain limn→∞Γn=0, that is, xn→x¯ as n→∞.

This completes the proof.□

Remark

Our iterative scheme has relatively low computational complexity compared to iterative schemes in [9] and [10].
The implementation of our algorithm does not need any prior information about the operator norm.
We can also use θj(x)=∥∇hj(x)∥2+∥∇l(x)∥2 instead of θj(x)=max{∥∇hj(x)∥,∥∇l(x)∥} perhaps the proof for the strong convergence theorem is almost similar. It is clear to see that max{∥∇hj(x)∥,∥∇l(x)∥}≤∥∇hj(x)∥2+∥∇l(x)∥2, for j∈Ψ.

Remark

One of the main advantages of our algorithm is that the algorithm can be used to solve problems that can be converted to the fixed point problem of firmly nonexpansive mapping. The following are some examples.

The split system of inclusion problem: Let Ti:H1→2H1, Uj:H2→2H2 be maximal monotone mappings for all i∈Φ and j∈Ψ. The split system of inclusion problem is to find x¯∈H1 such that
(43)0∈Ti(x¯),foralli∈Φ,0∈Uj(Ax¯),forallj∈Ψ.
Replacing the proximal mappings of the convex functions fi and gj in Algorithm 1 by the resolvent operators JλTi and JλUj to the maximal monotone operators, and following the method of proof in Theorem 3.2, we obtain an inertial extrapolation-type algorithm with a strong convergence result for approximation of solution of a consistent split system of inclusion problem (43); see the resolvent operator defined in (8).
The MSSFP: By taking fi=δCi and gj=δQj (the indicator functions) for i∈Φ, j∈Ψ, and replacing proxλfi by projection mapping PCi, and proxλgj by the projection mapping PQj in Algorithm 1, we obtain an inertial extrapolation-type algorithm with strong convergence for an approximation of solution of the MSSFP (1).
The split system of equilibrium problem: Let fi:H1×H1→ℝ and gj:H2×H2→ℝ be bifunctions, where i∈Φ, j∈Ψ. Assume each bifunction fi and gj satisfy Condition CO on H1 and H2, respectively. The split system of equilibrium problem involves finding x¯∈H1 such that

(44)fi(x¯,x)≥0,forallx∈H1,i∈Φ,gj(Ax¯,u)≥0,forallu∈H2,j∈Ψ.

Our result solves (44) by replacing the proximal mappings by the resolvent operators Tλfi and Tλgj in Algorithm 1 and then following the method of proof in Theorem 3.2; see the resolvent operator defined in Lemma 2.6.

It is worth mentioning that our approach also works for approximation of solution of SMP (4). Let Ω denote the solution set of (4), i.e., Ω={x¯∈argminf:Ax¯∈argming}, and assume that Ω is nonempty. Set l(x)=12∥(I−proxλf)x∥2, ∇l(x)=(I−proxλf)x, h(x)=12∥(I−proxλg)Ax∥2, ∇h(x)=A∗(I−proxλg)Ax and θ(x)=max{∥∇h(x)∥,∥∇l(x)∥}. The following is an immediate consequence of our result.

Algorithm 2

Initialization: Let V:H1→H1 be a contraction with constant γ. Choose x0, x1∈H1 and take arbitrary real numbers β and Θ^ such that 0≤β<1 and Θ^>0. Let {αn}, {εn}, {ρn} be real sequences satisfying the following conditions:

0<αn<1, limn→∞αn=0 and ∑n=1∞αn=∞;
εn>0 and εn=o(αn);
0<ρn<2 and liminfn→∞ρn(2−ρn)>0.

Step 1. Given the iterates xn−1 and xn (n≥1), choose βn such that 0≤βn≤β¯n, where

β¯n≔minβ,εn∥xn−1−xn∥,ifxn−1≠xn;β,otherwise.

Step 2. Evaluate yn=xn+βn(xn−xn−1).

Step 3. Evaluate μn=ρnh(yn)+l(yn)Θ2(yn), where Θ(yn)=Θ^,ifθ(yn)=0;θ(yn),otherwise.

Step 4. Evaluate zn=yn−12μn(∇h(yn)+∇l(yn)).

Step 5. Evaluate xn+1=αnV(yn)+(1−αn)zn.

Step 6. Set n≔n+1 and go to Step 1.

Corollary 1

The sequence{xn}generated by Algorithm 2 converges strongly tox¯∈Ω, wherex¯=PΩV(x¯).

Proof

Setting fi=f for all i∈Φ and gj=g for all j∈Ψ in Theorem 3.2, we obtain the desired result.□

Remark

The result in Corollary 1 is an improvement on inertial extrapolation-type algorithms in the sense that instead of weak convergence result proposed by Shehu and Iyiola in [25] we get strong convergence result.

4 Numerical results

In this section, we consider an example of the SSMP involving quadratic optimization problems. We study the behavior of our algorithm and compare with the proximal-type algorithms of [9] and [10]. The algorithm has been coded in MATLAB and is performed on a HP laptop with Intel(R) Core(TM) i5-7200U CPU @ 250GHz 2.70GHz and RAM 4.00GB.

Example

Consider the problem (7) for H1=ℝp, H2=ℝq, a linear transformation A:ℝp→ℝq and functions

fi(x)=12xTBix+xTDi(i∈Φ={1,…,N}),g1(u)=∥u∥qandg2(u)=∑k=1qh(uk),

where A is q×p non-zero matrix, Bi is an invertible symmetric positive semidefinite p×p matrix and Di is the zero vector in ℝp for all i∈Φ, u=(u1,…,uq)∈ℝq, ∥.∥q is the Euclidean norm in ℝq and h(uk)=max{|uk|−1,0} for k=1,…,q.

In this example, it is clear to see that Γ={0}. Now for λ=1, the proximal operators are given by

proxλfi(x)=(I+Bi)−1(x−Di)=(I+Bi)−1(x),i∈Φ,proxλg1(u)=1−1∥u∥qu,∥u∥q≥1,0,otherwise

and proxλg2(u)=(proxλh(u1),proxλh(u2),…,proxλh(uq)), where

proxλh(uk)=uk,if|uk|<1,sign(uk),if1≤|uk|≤2,sign(uk−1),if|uk|>2.

In this numerical experiment, we use N=3, p=q, A is identity p×p matrix and B1, B2 and B3 are randomly generated invertible symmetric positive semidefinite p×p matrices.

Experiment 1 (Studying numerical behavior of Algorithm 1):Figures 1, 2 and 3 and Tables 1 and 2 describe the numerical results of our algorithm for this example, where V:ℝp→ℝp given by Vx=γx and γ=0.5, αn=12nt, εn=1nr, ξn(j)=j3, β=0.9 and βn=β¯n for 0<t≤1, r>t, j∈Ψ={1,2}.

$Figure 1 For p=q=6p=q=6 and for randomly generated starting points x0{x}_{0} and x1{x}_{1}.$

Figure 1

For p=q=6 and for randomly generated starting points x0 and x1.

$Figure 2 For t=0.1t=0.1, r=0.4r=0.4 and for randomly generated starting points x0{x}_{0} and x1{x}_{1}.$

Figure 2

For t=0.1, r=0.4 and for randomly generated starting points x0 and x1.

$Figure 3 For t=0.2t=0.2, r=0.5r=0.5, p=q=500p=q=500, x1=δx0{x}_{1}=\delta {x}_{0}, where δ∈ℝ\delta \in {\mathbb{R}} and x0{x}_{0} is randomly generated starting point.$

Figure 3

For t=0.2, r=0.5, p=q=500, x1=δx0, where δ∈ℝ and x0 is randomly generated starting point.

Table 1

For randomly generated starting points x0 and x1

	p=q=3		p=q=20		p=q=80
	Iter(n)	CPU(s)	Iter(n)	CPU(s)	Iter(n)	CPU(s)
t=0.1, r=0.2	17	0.0127	14	0.0169	16	0.0269
t=0.95, r=3	21	0.0136	17	0.0185	21	0.0304
t=1, r=10	21	0.0129	16	0.0164	24	0.0299

Table 2

For t=0.55, r=8, p=q=4

x0=(1,2,3,4), x1=(−4,7,−5,10)			x0=(5,6,7,8), x1=(4,−7,5,−10)
Iter(n)	CPU(s)	∥xn−xn−1∥	Iter(n)	CPU(s)	∥xn−xn−1∥
1		12.2474	1		23.7486
2		5.6570	2		9.0177
3		3.5203	3		4.5572
4		1.9316	4		2.3349
5		1.1915	5		1.8997
.					.
.					.
.					.
24		2.5606×10−4	24		5.1148×10−4
25	0.0438	1.3608×10−4	25		3.6416×10−4
			26		3.1116×10−4
			27		2.4828×10−4
			28		2.0318×10−4
			29	0.0613	1.1041×10−4

Tables 1 and 2 illustrate the execution time in second (CPU(s)) and the number of iterations (Iter(n)) of our algorithm when applied to this particular example. The stopping criterion in Tables 1 and 2 is defined as ∥xn−xn−1∥∥x2−x1∥≤TOL=10−4.

Experiment 2 (Comparison): We now compare our result with non-inertial extrapolated (non-accelerated) proximal-type algorithms [9] (ProxAL-A) and [10] (ProxAL-B). For this purpose, we use the following data:

Algorithm 1:V:ℝp→ℝp given by Vx=γx and γ=0.5, αn=1n, εn=1n2, ξn(j)=j3 (j∈Ψ={1,2}), β=0.8, βn=β¯n and ρn=110.

ProxAL-A:V=F=I, μ=1, γ=0.5, δn(i)=i6 (i∈Φ={1,2,3}), ξn(j)=j3 (j∈Ψ={1,2}), αn=1n+1 and ρn=110, see [9].

ProxAL-B:δn(i)=i6 (i∈Φ={1,2,3}), ξn(j)=j3 (j∈Ψ={1,2}) and ρn=110, see [10].

Figures 4 and 5 along with Table 3 present the numerical results of our algorithm (Algorithm 1) in comparison with ProxAL-A and ProxAL-B. Figures 4 and 5 show the error=∥xn∥ versus number of iterations, while Table 2 shows the CPU time exclusion (CPU(s)) and the number of iterations (Iter(n)) of Algorithm 1, ProxAL-A and ProxAL-B for the stopping criteria ∥xn−xn−1∥∥x2−x1∥≤TOL=10−3.

$Figure 4 For p=q=4p=q=4 and x0=(1,1,1,1){x}_{0}=(1,1,1,1), x1=10x0{x}_{1}=10{x}_{0}.$

Figure 4

For p=q=4 and x0=(1,1,1,1), x1=10x0.

$Figure 5 For p=q=80p=q=80 and x0=(10,…,10){x}_{0}=(10,\ldots ,10), x1=10x0{x}_{1}=10{x}_{0}.$

Figure 5

For p=q=80 and x0=(10,…,10), x1=10x0.

Table 3

For x0,x1∈ℝp with x0=(−100,−100,…,−100) and x1=−2x0

	Algorithm 1		ProxAlg-A		ProxAlg-B
Dimension	Iter(n)	CPU(s)	Iter(n)	CPU(s)	Iter(n)	CPU(s)
p=q=2	7	0.0354	7	0.0401	9	0.0379
p=q=10	12	0.0441	15	0.0616	21	0.1775
p=q=50	27	0.1908	29	0.2168	29	0.1921

From this preliminary numerical experiment, we observe that our algorithm crucially depends on step sizes, starting points and dimensions. Moreover, our proposed algorithm is efficient and easy to implement and outperforms the proposed algorithms in [9] and [10].

5 Conclusions

In this article, we introduce a strong convergence theorem for an inertial extrapolation-type algorithm for solving a SSMP (7). The problem we considered in this article is general for many of the problems considered in the literature concerning approximation of an unconstrained minimization problem, see for example [25,26,27,28,24,23]. Our result can also be applied to find a solution of the split system of inclusion problem, the MSSFP, and the split system of equilibrium problem. Furthermore, our result improves an inertial extrapolation-type algorithm proposed in [25] and also improves and accelerates algorithms in [9,10].

References

[1] Y. Censor , T. Elfving , N. Kopf , and T. Bortfeld , The multiple-sets split feasibility problem and its applications for inverse problems, Inverse Problems 21 (2005), no. 6, 2071–2084, https://doi.org/10.1088/0266-5611/21/6/017 .10.1088/0266-5611/21/6/017Search in Google Scholar

[2] Y. Censor and T. Elfving , A multiprojection algorithm using Bregman projections in a product space, Numer. Algorithms 8 (1994), no. 2, 221–239, https://doi.org/10.1007/BF02142692 .10.1007/BF02142692Search in Google Scholar

[3] Y. Dang , Y. Gao , and L. Li , Inertial projection algorithms for convex feasibility problem, J. Syst. Eng. Electron. 23 (2012), no. 5, 734–740, https://doi.org/10.1109/JSEE.2012.00090 .10.1109/JSEE.2012.00090Search in Google Scholar

[4] G. López , V. Martín-Márquez , F. Wang , and H. K. Xu , Solving the split feasibility problem without prior knowledge of matrix norms, Inverse Problems 28 (2012), no. 8, 85–104, https://doi.org/10.1088/0266-5611/28/8/085004 .10.1088/0266-5611/28/8/085004Search in Google Scholar

[5] B. Qu and N. Xiu , A note on the CQ algorithm for the split feasibility problem, Inverse Problems 21 (2005), no. 5, 1655–1665, https://doi.org/10.1088/0266-5611/21/5/009 .10.1088/0266-5611/21/5/009Search in Google Scholar

[6] A. Latif , J. Vahidi , and M. Eslamian , Strong convergence for generalized multiple-set split feasibility problem, Filomat 30 (2016), no. 2, 459–467, https://doi.org/10.2298/FIL1602459L .10.2298/FIL1602459LSearch in Google Scholar

[7] E. Masad and S. Reich , A note on the multiple-set split convex feasibility problem in Hilbert space, J. Nonlinear. Convex Anal. 8 (2007), no. 3, 367–371.Search in Google Scholar

[8] H. K. Xu , A variable Krasnosel’skii–Mann algorithm and the multiple-set split feasibility problem, Inverse Problems 22 (2006), no. 6, 2021–2034, https://doi.org/10.1088/0266-5611/22/6/007 .10.1088/0266-5611/22/6/007Search in Google Scholar

[9] A. G. Gebrie and R. Wangkeeree , Proximal method of solving split system of minimization problem, J. Appl. Math. Comput. 63 (2020), 107–132, https://doi.org/10.1007/s12190-019-01310-w .10.1007/s12190-019-01310-wSearch in Google Scholar

[10] A. G. Gebrie and R. Wangkeeree , An iterative scheme for solving split system of minimization problems, J. Comput. Anal. Appl. 28 (2020), no. 6, 968–980.Search in Google Scholar

[11] A. G. Gebrie and R. Wangkeeree , Parallel proximal method of solving split system of fixed point set constraint minimization problems, Rev. R. Acad. Cienc. Exactas Fıs. Nat. Ser. A Mat. RACSAM 114 (2020), no. 1, https://doi.org/10.1007/s13398-019-00758-6 .10.1007/s13398-019-00758-6Search in Google Scholar

[12] Y. C. Tang and L. W. Liu , Several iterative algorithms for solving the split common fixed point problem of directed operators with applications, Optimization 65 (2016), no. 1, 53–65, https://doi.org/10.1080/02331934.2014.984708 .10.1080/02331934.2014.984708Search in Google Scholar

[13] Y. Censor and A. Segal , The split common fixed point problem for directed operators, J. Convex Anal. 16 (2009), no. 2, 587–600.Search in Google Scholar

[14] A. Abkar and E. Shahrosvand , The split common fixed point problem of two infinite families of demicontractive mappings and the split common null point problem, Filomat 31 (2017), no. 12, 3859–3874, https://doi.org/10.2298/FIL1712859A .10.2298/FIL1712859ASearch in Google Scholar

[15] M. M. Alves and B. F. Svaiter , A proximal-Newton method for unconstrained convex optimization in Hilbert spaces, Optimization 67 (2018), no. 1, 67–82, https://doi.org/10.1080/02331934.2017.1389942 .10.1080/02331934.2017.1389942Search in Google Scholar

[16] P. E. Gill and W. Murray , Newton-type methods for unconstrained and linearly constrained optimization, Math. Program. 7 (1974), no. 1, 311–350, https://doi.org/10.1007/BF01585529 .10.1007/BF01585529Search in Google Scholar

[17] R. T. Rockafellar and R. J. Wets , Variational Analysis, Springer-Verlag, Berlin, Heidelberg, 2009.Search in Google Scholar

[18] B. Martinet , Régularisation d’inéquations variationnelles par approximations successives, R.I.R.O. 4 (1970), no. R3, 154–158, https://doi.org/10.1051/m2an/197004R301541 .10.1051/m2an/197004R301541Search in Google Scholar

[19] B. Martinet , Détermination approchée d’un point fixe d’une application pseudo-contractante, CR Acad. Sci. Paris. 274 (1972), no. 2, 163–165.Search in Google Scholar

[20] R. T. Rockafellar , Monotone operators and the proximal point algorithm, SIAM J. Control Optim. 14 (1976), no. 5, 877–898, https://doi.org/10.1137/0314056 .10.1137/0314056Search in Google Scholar

[21] A. Moudafi and B. S. Thakur , Solving proximal split feasibility problems without prior knowledge of operator norms, Optim. Lett. 8 (2014), no. 7, 2099–2110, https://doi.org/10.1007/s11590-013-0708-4 .10.1007/s11590-013-0708-4Search in Google Scholar

[22] J. M. Hendrickx and A. Olshevsky , Matrix p-norms are NP-hard to approximate if p ≠ 1,2,∞, SIAM J. Matrix Anal. Appl. 31 (2010), no. 5, 2802–2812.10.1137/09076773XSearch in Google Scholar

[23] M. Abbas , M. AlShahrani , Q. H. Ansari , O. S. Iyiola , and Y. Shehu , Iterative methods for solving proximal split minimization problems, Numer. Algorithms 78 (2018), no. 1, 193–215, https://doi.org/10.1007/s11075-017-0372-3 .10.1007/s11075-017-0372-3Search in Google Scholar

[24] Y. Shehu , G. Cai , and O. S. Iyiola , Iterative approximation of solutions for proximal split feasibility problems. Fixed Point Theory Appl. 2015 (2015), art. 123, https://doi.org/10.1186/s13663-015-0375-5 .10.1186/s13663-015-0375-5Search in Google Scholar

[25] Y. Shehu and O. S. Iyiola , Convergence analysis for the proximal split feasibility problem using an inertial extrapolation term method, J. Fixed Point Theory Appl. 19 (2017), no. 4, 2483–2510, https://doi.org/10.1007/s11784-017-0435-z .10.1007/s11784-017-0435-zSearch in Google Scholar

[26] Y. Shehu and O. S. Iyiola , Strong convergence result for proximal split feasibility problem in Hilbert spaces, Optimization 66 (2017), no. 12, 2275–2290, https://doi.org/10.1080/02331934.2017.1370648 .10.1080/02331934.2017.1370648Search in Google Scholar

[27] Y. Shehu and O. S. Iyiola , Accelerated hybrid viscosity and steepest-descent method for proximal split feasibility problems, Optimization 67 (2018), no. 4, 475–492, https://doi.org/10.1080/02331934.2017.1405955 .10.1080/02331934.2017.1405955Search in Google Scholar

[28] Y. Shehu and O. S. Iyiola , Nonlinear iteration method for proximal split feasibility problems, Math. Meth. Appl. Sci. 41 (2018), no. 2, 781–802.10.1002/mma.4644Search in Google Scholar

[29] Y. Shehu and F. U. Ogbuisi , Convergence analysis for proximal split feasibility problems and fixed point problems, J. Appl. Math. Comp. 48 (2015), no. 1–2, 221–239, https://doi.org/10.1007/s12190-014-0800-7 .10.1007/s12190-014-0800-7Search in Google Scholar

[30] B. T. Polyak , Some methods of speeding up the convergence of iteration methods, USSR Comput. Math. Math. Phys. 4 (1964), no. 5, 1–17, https://doi.org/10.1016/0041-5553(64)90137-5 .10.1016/0041-5553(64)90137-5Search in Google Scholar

[31] H. Attouch , J. Peypouquet , and P. Redont , A dynamical approach to an inertial forward-backward algorithm for convex minimization, SSIAM J. Optim. 24 (2014), no. 1, 232–256, https://doi.org/10.1137/130910294 .10.1137/130910294Search in Google Scholar

[32] A. Beck and M. Teboulle , A fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM J. Imaging Sci. 2 (2009), no. 1, 183–202, https://doi.org/10.1137/080716542 .10.1137/080716542Search in Google Scholar

[33] C. Chen , R. H. Chan , S. Ma , and J. Yang , Inertial proximal ADMM for linearly constrained separable convex optimization, SIAM J. Imaging Sci. 8 (2015), no. 4, 2239–2267, https://doi.org/10.1137/15100463X .10.1137/15100463XSearch in Google Scholar

[34] P. Ochs , T. Brox , and T. Pock , iPiasco: Inertial proximal algorithm for strongly convex optimization, J. Math. Imaging Vision. 53 (2015), no. 2, 171–181, https://doi.org/10.1007/s10851-015-0565-0 .10.1007/s10851-015-0565-0Search in Google Scholar

[35] H. H. Bauschke and P. L. Combettes , Convex Analysis and Monotone Operator Theory in Hilbert Spaces, Springer International, New York, 2011.10.1007/978-1-4419-9467-7Search in Google Scholar

[36] H. Brezis , Operateurs Maximaux Monotones: Et Semi-Groupes De Contractions Dans Les Espaces De Hilbert, North-Holland Mathematics Studies, vol. 5 , American Elsevier Publishing Company, Inc., New York, 1973.Search in Google Scholar

[37] R. T. Rockafellar , On the maximality of sums of nonlinear monotone operators, Trans. Amer. Math. Soc. 149 (1970), no. 1, 75–88, https://doi.org/10.2307/1995660 .10.1090/S0002-9947-1970-0282272-5Search in Google Scholar

[38] B. Lemaire , Which fixed point does the iteration method select? , in: P. Gritzmann , R. Horst , E. Sachs , R. Tichatschke , (eds), Recent Advances in Optimization, Lecture Notes in Economics and Mathematical Systems, vol. 452 , Springer, Berlin, Heidelberg, pp. 154–167, https://doi.org/10.1007/978-3-642-59073-3_11 10.1007/978-3-642-59073-3_11Search in Google Scholar

[39] H. K. Xu , Iterative algorithms for nonlinear operators, Lond. Math. Soc. 66 (2002), no. 1, 240–256.10.1112/S0024610702003332Search in Google Scholar

[40] P. E. Maingé , Strong convergence of projected subgradient methods for nonsmooth and nonstrictly convex minimization, Set-Valued Anal. 16 (2008), no. 7–8, 899–912, https://doi.org/10.1007/s11228-008-0102-z .10.1007/s11228-008-0102-zSearch in Google Scholar

[41] E. Blum and W. Oettli , From optimization and variational inequalities to equilibrium problems, Math. Student 63 (1994), 123–145.Search in Google Scholar

[42] P. L. Combettes and S. A. Hirstoaga , Equilibrium programming in Hilbert spaces, J. Nonlinear Convex Anal. 6 (2005), no. 1, 117–136.Search in Google Scholar

Received: 2020-07-18

Revised: 2020-10-10

Accepted: 2020-10-26

Published Online: 2020-12-31

This work is licensed under the Creative Commons Attribution 4.0 International License.

Strong convergence of an inertial extrapolation method for a split system of minimization problems

Abstract

1 Introduction

Theorem 1.1

2 Preliminary

Lemma 2.1

Definition 2.1

Lemma 2.2

Lemma 2.3

Definition 2.2

Lemma 2.4

Lemma 2.5

Lemma 2.6

3 Main result

Remark

Assumption 1

Algorithm 1

Remark

Remark

Remark

Lemma 3.1

Proof

Theorem 3.2

Proof of Theorem 3.2

Remark

Remark

Algorithm 2

Corollary 1

Proof

Remark

4 Numerical results

Example

5 Conclusions

References

Journal and Issue

Articles in the same Issue