EUROPEAN JOURNAL OF PURE AND APPLIED MATHEMATICS
Vol. 15, No. 4, 2022, 2043-2053
ISSN 1307-5543 – ejpam.com
Published by New York Business Global

A New Type Coefficient Conjugate on the Gradient
Methods for Impulse Noise Removal in Images

Basim A. Hassan1,∗, Haneen A. Alashoor2

1 Department of Mathematics, College of Computers Sciences and Mathematics,
University of Mosul, Iraq

Abstract. The conjugate gradient methods are outstanding by choosing a suitable for coefficient
conjugate. In this paper, a modified version of the conjugate gradient algorithm suggested by
Hideaki and Yasushi [4] is proposed in order to show that the new method is globally convergent,
under standard assumptions. To exemplify the efficiency of the new method, its performance is
examined for impulse noise removal in images.

2020 Mathematics Subject Classifications: 90C30, 65K05, 49M37

Key Words and Phrases: Derivative coefficient conjugate gradient, Convergence property, Im-
age restoration problems

1. Introduction

Image declaring [16] from noisy data is a fundamental problem in image processing.
Two-phase approach, consists of two phases. Firstly, accurate detection of the location of
impulse noise using a median-type filter (AMF) [17]. Let x be the true image with M-by-N
pixels, and xij

M,N
i,j=1 denote the gray level of x, y signifying the observed noisy image of x

corrupted by the salt-and-pepper noise, ȳ is defined by the image obtained byapplying the
adaptive median filter method to the noisy image y in the first phase. Secondly, recovering
the noise pixels by minimizing the following functional:

fα(u) =
∑

(i,j)∈N

[|ui,j − yi,j |+
β

2
(S1

i,j + S2
i,j)] (1)

Where ui,j = ⌊ui,j⌋(i,j)∈N is a column vector of length |N |, β is the regularization param-
eter and:

S1
i,j = 2

∑
(m,n)∈Pi,j∩Nc

φα(ui,j − ym,n), S2
i,j =

∑
(m,n)∈Pi,j∩N

φα(ui,j − ym,n) (2)

∗Corresponding author.
DOI: https://doi.org/10.29020/nybg.ejpam.v15i4.4579

Email addresses: basimah@uomosul.edu.iq (B. A. Hassan), haneen19921006@gmail.com (H. A. Alashoor)

https://www.ejpam.com 2043 © 2022 EJPAM All rights reserved.



B. A. Hassan, H. A. Alashoor / Eur. J. Pure Appl. Math, 15 (4) (2022), 2043-2053 2044

The noise candidate indices set N c = {(i, j) ∈ A/ȳi,j ̸= yi,j and yi,j = smax or smin}, smax

is the maximum of the noisy pixel and smin denotes the minimum of the noisy pixel.
A = 1, 2, 3, ...,M × 1, 2, 3, ..., N , Vi,j = (Vi,j ∩ N c) ∪ (Vi,j ∩ N) is the neighborhood of
(i, j), and φα is an edge preserving potential function having the parameter α, example
of such φα =

√
α+ x2, α > 0. Similar optimization problems arise with non-smooth

regularizations, where Fα(u) is of the form (1) with S1
i,j +S2

i,j smooth and |ui,j −yi,j | non-
smooth at zero. In formula, the function which is minimized is a half-quadratic smooth
approximation of Fα(u) as:

fα(u) =
∑

(i,j)∈N

[(2× S1
i,j + S2

i,j)] (3)

More details can be found in [16],[17].
The minimization of formula (3) has been accomplished by the method of conjugated

gradients such as:
f(x∗) = min

x∈RN
f(u). (4)

In formula [15], the k-th iteration, a step-length αk is obtained by a line search tech-
nique and the next iterate is set to:

uk+1 = uk + αkdk. (5)

If f is a convex quadratic, its one-dimensional minimizer along the ray uk + αkdk can be
computed analytically, and is given by:

αk = −
gTk dk

dTkQdk
. (6)

For general non-linear functions, it is necessary to use an iterative procedure. More details
can be found in formula [14].

In the convergence analysis and implementations of conjugate gradient methods, the
Wolfe condition is often used to find the step length αk satisfying:

f(uk + akdk) ≤ f(uk) + δαkg
T
k dk (7)

dTk g(uk + akdk) ≥ σ dTk gk (8)

where 0 < δ < σ < 1. More details can be found in [13].
The conjugate gradient direction for the next iteration has the following form:

dk+1 = −gk+1 + βkdk (9)

where βk is a scalar. Two famous ways of choosing βk are:

βFR
k =

∥gk+1|2

∥gk|2
, βDY

k+1 =
∥gk+1|2

dTk yk
(10)



B. A. Hassan, H. A. Alashoor / Eur. J. Pure Appl. Math, 15 (4) (2022), 2043-2053 2045

These were given by Fletcher-Reeves (FR) method [4], and the Dai - Yuan (DY) method [2]
independently. Its algorithms have important properties including the globally convergent
property.]

Recently, Hideaki and Yasushi [11] and Basim [8] proposed conjugate gradient methods
which significantly differ in using both their gradient and function values with higher
accuracy in the approximation of the curvature, in which the parameters are specified as
follows:

βHY
k =

∥gk+1|2

2(fk − fk+1)/ak
, βB

k =
∥gk+1|2

(fk − fk+1)/ak − gTk dk/2
(11)

These algorithms are significantly efficient in their computational efficiency also. The
reported improved performance for this modification for unconstrained problems is a
quadratic model in order to optimize the benefits of the original conjugate gradient meth-
ods.

The main goal of this study is derivation of the new coefficient conjugate and to think
differently of the denominator dTkGvk based on the quadratic model. Our method is found
to be numerically coherent and also efficient in image restoration.

2. Deriving new coefficient conjugate for conjugate gradient methods

We will assume that f is quadratic and exact line search are being used. This is a
studies on the behaviors of one of the family of conjugate gradient optimization methods,
which was introduced by Hideaki and Yasushi in 2011. Hideaki and Yasushi choice of βk
is:

βk =
gTk+1Qsk

dTkQsk
(12)

where Q is Hessian matrix and where βk is satisfies the conjugacy condition:

dTk+1Qdk = 0 (13)

To modify this method we will introduce a good approximation to the dTkQsk. So by
Taylor formula we have:

f(u) = f(uk+1) + gTk+1(u− uk+1) +
1

2
(u− uk+1)

TQ(uk)(u− uk+1) (14)

Define the gradient step:

gk+1 = gk +Q(uk)sk (15)

Note that equation (14) and equation (15) result that:

1/2sTkQ(uk)sk = (fk+1 − fk − gTk sk) (16)



B. A. Hassan, H. A. Alashoor / Eur. J. Pure Appl. Math, 15 (4) (2022), 2043-2053 2046

Now, by equation (16) and equation (6) we can see that:

dTkQ(uk)sk = (fk+1 − fk)/ak − 3/2dTk gk (17)

Using (12) and (17), we obtain the following formula for computing βk:

βk =
gTk+1yk

(fk+1 − fk)/αk − 3/2dTk gk
(18)

On the other hand, using (6) in (17) and putting in (12), then got another formula:

βk =
gTk+1yk

(fk+1 − fk)/αk + 3/2dTk yk
(19)

Since f is quadratic function and exact line search is employed, then:

βk =
∥gk+1|2

(fk+1 − fk)/αk − 3/2dTk gk
(20)

and

βk =
∥gk+1|2

(fk+1 − fk)/αk + 3/2dTk yk
(21)

Given our formula for modified HY formula, so-called BNC and BTC.
At this point, we describe a new algorithm, called BNC and BTC.
Algorithms BNC and BTC.

Stage 1. An initial point u1. Set d1 = −g1. If ∥g1| = 10−6, then stop.
Stage 2. Determine the αk > 0 satisfying the Wolfe conditions (7) and (8).
Stage 3. Let xk+1 = xk + αkdk and gk+1 = g(xk+1). If ∥gk+1| = 10−6, then stop.
Stage 4. Compute βk by the formulae (20-21), then generate dk+1 by (9).
Stage 5. Set k = k + 1 and continue with step 2.

The nice property for any a good algorithm should accomplish the sufficient descent
condition. Specially, in this study will provide additional property are very vital in the
convergence proving. Summarizing the above discussion, the following theorem is ob-
tained.

Theorem 1. Let xk and dk be generated by BNC method, then we obtain:

dTk+1gk+1 < 0 and dTk+1gk+1 = βkd
T
k gk (22)

Proof. It is visible that dk = −gk satisfies dT1 g1 < 0. Suppose that dTk gk < 0 for any k.
Directly from (11) and (21) we conclude that:

dTk+1gk+1 = −gTk+1gk+1 + βkd
T
k gk+1

= −βk((fk+1 − fk)/ak − 3/2dTk gk) + βkd
T
k gk+1

(23)



B. A. Hassan, H. A. Alashoor / Eur. J. Pure Appl. Math, 15 (4) (2022), 2043-2053 2047

Since the gradient of the function (14) is (15), we can conclude that:

dTk+1gk+1 = βkd
T
k gk (24)

Because dTk gk < 0, then we obtain:

dTk+1gk+1 < 0 (25)

so the proof is completed. A similar result holds for the BTC formula.

3. Convergence Analysis

In order to analyze the global convergence of the BDC and BTC methods, we begin
by impose the following assumptions :

[1] The level set “D = u/f(u) ≤ f(u0)” is bounded, where u0 is the starting point.

[2] The gradientis is Lipschitz continuous in some neighborhood D which contains L0;
i.e. there exist L, such that:

∥g(υ)− g(ω)| ≤ L∥υ − ω|, ∀υ, ω ∈ D. (26)

For more details see [2],[5].
In [18], Zoutendijk introduced the general result are important feature in proving the

convergence results.

Lemma 1. Let the Assumptions (1) and (2) holds true. If the direction dk is descent and
the sequence generated by (5) where αk satisfies (7) and (8), then:

∞∑
k=1

(gTk dk)
2

∥dk|2
< ∞. (27)

Theorem 2. Suppose that assumptions (1) and (2) hold. If formula βk satisfies (21),
then Then we have:

lim
k→∞

inf |gk| = 0. (28)

Proof. The prove is by induction ∥gk| ≠ 0 for all k ∈ n. Rewriting (9) as dk+1+gk+1 =
βkdk. Accordingly, squaring both sides, we have:

|dk+1|2 + |gk+1|2 + 2dTk+1gk+1 = (βk)
2|dk|2 (29)

Using (23) to (27) implies:

|dk+1|2 =
(dTk+1gk+1)

2

(dTk gk)
2

|dk|2 − 2dTk+1gk+1 − |gk+1|2 (30)



B. A. Hassan, H. A. Alashoor / Eur. J. Pure Appl. Math, 15 (4) (2022), 2043-2053 2048

Dividing both sides of (28) by (dTk+1gk+1)
2, we get:

|dk+1|2

(dTk+1gk+1)2
=

|dk|2

(dTk gk)
2
− |gk+1|2

(dTk+1gk+1)2
− 2

dTk+1gk+1

≤ |dk|2

(dTk gk)
2
− (

|gk+1|
(dTk+1gk+1)

+
1

|gk+1|2
) +

1

|gk+1|2

≤ |dk|2

(dTk gk)
2
+

1

|gk+1|2

(31)

Hence, we obtained:

|dk+1|2

(dTk+1gk+1)2
≤

k+1∑
i=1

1

|gi|2
(32)

Assume that there exists c1 > 0 such that |gk| ≥ c1 for all k ∈ n. Then:

|dk+1|2

(dTk+1gk+1)2
<

k + 1

c21
(33)

Equation (31) shows that:

∞∑
k=1

(gTk dk)
2

|dk|2
= ∞ (34)

Based on Lemma 1, we get limk→∞ inf |gk| = 0 holds. A similar result holds for the
BTC formula.

4. Numerical results

The main goal we carry out some numerical experiments for minimization of 4 test
images taken from [3]. To test and compare the computation effect of the proposed
BNC, BTC in this paper with FR method whose results be given by [4]. There are
different methods than this methods that we can see in [6, 7, 9, 10, 12]. In the numerical
experiments, BNC, BTC and FR methods uses the following parameters: δ = 0.0001 and
σ = 0.5.

Numerical results of the BNC and FR are listed in Table 1. Here denotes the NI/NF/PSNR
denote the number of iteration, function evaluations and PSNR (peak signal to noise ratio),
which is defined as:

PSNR = 10log10
2552

1
MN

∑
i,j (u

r
i,j − u∗i,j)

2
(35)

where uri,j and u∗i,j denote the pixel values of the restored image and the original image,
respectively. We stop the iterations, if:



B. A. Hassan, H. A. Alashoor / Eur. J. Pure Appl. Math, 15 (4) (2022), 2043-2053 2049

|f(uk)− f(uk−1)|
|f(uk)|

≤ 10−4 and |f(uk)| ≤ 10−4(1 + |f(uk)|) (36)

For more details see [1],[3].

Table 1: Numerical results for the NI , NF and PSNR.

As we can see from Table (1), our proposed algorithm is competitive and promising
has a very great advantage over the FR method in term of the the number ofiterations
and the number of function evaluations.

Figure 1: From left to right: 50, 70, 90% noise, FR method, BNC and BTC method 256× 256 Lena image.



B. A. Hassan, H. A. Alashoor / Eur. J. Pure Appl. Math, 15 (4) (2022), 2043-2053 2050



REFERENCES 2051

Figure 2: From left to right: 50, 70, 90% noise, FR method, BNC and BTC method 512 × 512 Cameraman
image.

5. Conclusions

We presented a powerful conjugate gradient technique. In addition to meeting the
adequate descent criterion, the proposed approach is globally convergent. According to
numerical findings, the approach operates well in practice and is superior than the widely
used FR method. We also looked at our approach’s aptitude for resolving several practical
problems. In this manner, a typical issue from applications for image processing was taken
into account. We demonstrated the acceptability of the image that our approach restored.

References

[1] Jian-Feng Cai, Raymond Chan, and Benedetta Morini. Minimization of an edge-
preserving regularization functional by conjugate gradient type methods. In Image
Processing Based on Partial Differential Equations, pages 109–122. Springer, 2007.

[2] Yu-Hong Dai and Yaxiang Yuan. A nonlinear conjugate gradient method with a
strong global convergence property. SIAM Journal on optimization, 10(1):177–182,
1999.

[3] Yuhong Dai, Jiye Han, Guanghui Liu, Defeng Sun, Hongxia Yin, and Ya-xiang Yuan.
Convergence properties of nonlinear conjugate gradient methods. SIAM Journal on
Optimization, 10(2):345–358, 2000.



REFERENCES 2052

[4] Reeves Fletcher and Colin M Reeves. Function minimization by conjugate gradients.
The computer journal, 7(2):149–154, 1964.

[5] William W Hager and Hongchao Zhang. A new conjugate gradient method with guar-
anteed descent and an efficient line search. SIAM Journal on optimization, 16(1):170–
192, 2005.

[6] BA Hassan and RM Sulaiman. A new class of self-scaling for quasi-newton method
based on the quadratic model. Indonesian Journal of Electrical Engineering and
Computer Science, 21(3):1830–1836, 2021.

[7] Basim A Hassan. A modified quasi-newton methods for unconstrained optimization.
Pure Appl. Math, 42(2019):504–511, 2019.

[8] Basim A Hassan. A new formula for conjugate parameter computation based on
the quadratic model. Indonesian Journal of Electrical Engineering and Computer
Science, 3(3):954–961, 2019.

[9] Basim A Hassan. A new type of quasi-newton updating formulas based on the new
quasi-newton equation. Numerical Algebra, Control & Optimization, 10(2):227, 2020.

[10] Basim A Hassan, Kanikar Muangchoo, Fadhil Alfarag, Abdulkarim Hassan Ibrahim,
and Auwal Bala Abubakar. An improved quasi-newton equation on the quasi-newton
methods for unconstrained optimizations. Indonesian Journal of Electrical Engineer-
ing and Computer Science, 22(2):389–397, 2020.

[11] Hideaki Iiduka and Yasushi Narushima. Conjugate gradient methods using value of
objective function for unconstrained optimization. Optimization Letters, 6(5):941–
955, 2012.

[12] Hawraz N Jabbar and Basim A Hassan. Two-versions of descent conjugate gradient
methods for large-scale unconstrained optimization. Indonesian Journal of Electrical
Engineering and Computer Science, 22(3):1643, 2021.

[13] Xian-zhen Jiang and Jin-bao Jian. A sufficient descent dai–yuan type nonlinear conju-
gate gradient method for unconstrained optimization problems. Nonlinear Dynamics,
72(1):101–112, 2013.

[14] Jorge Nocedal and Stephen J Wright. Numerical optimization, springer series in
operations research. Siam J Optimization, 2006.

[15] Elijah Polak and Gerard Ribiere. Note sur la convergence de méthodes de direc-
tions conjuguées. Revue française d’informatique et de recherche opérationnelle. Série
rouge, 3(16):35–43, 1969.

[16] Wei Xue, Junhong Ren, Xiao Zheng, Zhi Liu, and Yueyong Liang. A new dy conjugate
gradient method and applications to image denoising. IEICE TRANSACTIONS on
Information and Systems, 101(12):2984–2990, 2018.



REFERENCES 2053

[17] Gaohang Yu, Jinhong Huang, and Yi Zhou. A descent spectral conjugate gradient
method for impulse noise removal. Applied mathematics letters, 23(5):555–560, 2010.

[18] G Zoutendijk. Nonlinear programming, computational methods. Integer and nonlin-
ear programming, pages 37–86, 1970.